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Why  GAO  Did  This  Study 

This  is  GAO’s  annual  assessment  of 
DOD  weapon  system  acquisitions,  an 
area  that  is  on  GAO’s  High-Risk  List. 
DOD  and  Congress  have  taken 
meaningful  steps  to  improve  the 
acquisition  of  major  weapon  systems, 
yet  many  programs  are  still  falling  short 
of  cost  and  schedule  estimates. 
Moreover,  GAO  has  emphasized  the 
importance  of  completing 
developmental  testing  before  programs 
enter  production  and  pointed  out  risks 
associated  with  such  concurrent 
testing  and  production.  With  expected 
budgetary  pressures,  DOD  cannot 
afford  to  miss  opportunities  to  address 
inefficiencies  in  these  programs  to  free 
up  resources  for  higher  priority  needs. 

This  report  responds  to  a  mandate  in 
the  joint  explanatory  statement  to  the 
DOD  Appropriations  Act,  2009.  It 
includes  observations  on  (1)  the  cost 
and  schedule  performance  of  DOD’s 
2013  portfolio  of  80  major  defense 
acquisition  programs;  (2)  the 
knowledge  attained  at  key  junctures  in 
the  acquisition  process  for  38 
programs  that  were  in  development  or 
early  production;  and  (3)  key 
acquisition  reform  initiatives  and 
program  concurrency.  To  develop  the 
observations  in  this  report,  GAO 
analyzed  cost,  schedule,  and  quantity 
data  from  DOD’s  Selected  Acquisition 
Reports.  GAO  also  collected  data  from 
program  offices  on  technology,  design, 
and  manufacturing  knowledge;  the  use 
of  knowledge-based  acquisition 
practices;  and  the  implementation  of 
acquisition  reforms  and  initiatives. 

In  commenting  on  a  draft  of  this  report, 
DOD  cited  its  commitment  to 
acquisition  excellence  and  its  positive 
effect  on  programs. 

View  GAO-14-340SP.  For  more  information, 
contact  Michael  J.  Sullivan  at  (202)  512-4841 
or  sullivanm@gao.gov. 
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What  GAO  Found 

Over  the  past  year,  the  overall  size  of  DOD’s  major  defense  acquisition  program 
portfolio  decreased,  from  85  programs  to  80,  while  the  estimated  cost  has 
increased  by  $14.1  billion.  The  average  time  to  deliver  initial  capability  to  the 
warfighter  also  increased  by  2  months.  The  slight  cost  increase  can  be  attributed 
to  the  addition  of  one  program,  the  Evolved  Expendable  Launch  Vehicle.  This 
furthers  a  trend  for  decreased  portfolio  size  seen  for  the  past  three  years. 
Although  the  overall  cost  of  the  2013  portfolio  increased,  50  of  the  80  programs 
decreased  costs,  and  64  percent  of  programs  increased  their  buying  power. 
There  are  still  some  programs  that  have  performed  poorly,  both  over  the  past 
year  and  in  the  longer  term.  Fifty-five  percent  of  the  current  portfolio  funding  has 
been  appropriated,  leaving  approximately  $682  billion  needed  for  future  funding. 
About  forty-five  percent  of  this  remaining  funding  represents  cost  growth  from 
initial  estimates,  a  clear  indicator  that  DOD  needs  to  do  more  to  control  cost 
growth. 


DOD  Portfolio  Cost  and  Size,  2004-2013 

Fiscal  year  2014  dollars  (in  billions)  Number  of  programs 


Total  portfolio  acquisiton  cost 


Number  of  programs 


Source:  GAO  analysis  of  DOD  data. 

Note:  The  2009  portfolio  is  excluded  as  the  appropriate  data  was  not  available  for  review. 


Most  of  the  38  programs  GAO  assessed  this  year  are  not  yet  fully  following  a 
knowledge-based  acquisition  approach.  This  held  true  for  the  seven  programs 
that  passed  through  one  of  three  key  decision  points  in  the  past  year.  Each 
implemented  some  knowledge  based  practices  but  practices — such  as  fully 
maturing  technologies  prior  to  development  start  and  bringing  all  manufacturing 
processes  under  control — were  not  implemented.  As  a  result,  many  of  the  38 
programs  will  carry  unwanted  risk  into  subsequent  phases  of  acquisition  that 
could  result  in  cost  growth  or  schedule  delays. 


Implementation  of  the  reform  initiatives  on  the  38  programs  assessed  above,  and 
1 8  that  are  not  yet  major  defense  acquisition  programs,  GAO  analyzed  varies. 
Specifically,  while  the  department  has  been  successful  in  implementing  its 
“should  cost”  initiative  to  reduce  contract  costs  and  established  configuration 
steering  boards,  the  establishment  of  limits  on  total  program  costs  (i.e. 
affordability  constraints)  and  the  need  for  greater  competition  are  not  being 
implemented  to  the  same  degree.  In  addition,  many  programs  continue  to  commit 
to  production  before  completing  developmental  testing. 

_ United  States  Government  Accountability  Office 


Contents 


Foreword  i 


Letter  3 

Observations  on  the  Cost  and  Schedule  Performance  of  DOD’s 

2013  Major  Defense  Acquisition  Program  Portfolio  5 

Observations  from  Our  Assessment  of  Knowledge  Attained  by 

Programs  at  Key  Acquisition  Junctures  22 

Observations  about  DOD’s  Implementation  of  Key  Acquisition 

Reform  Initiatives  and  Program  Concurrency  35 

Assessments  of  Individual  Programs  47 

Statement  on  Small  Business  Participation  51 

Two-Page  Assessments  of  Individual  Programs 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II)  53 

Air  and  Missile  Defense  Radar  (AMDR)  55 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K)  57 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000)  59 

Enhanced  Polar  System  (EPS)  61 

Evolved  Expendable  Launch  Vehicle  (EELV)  63 

Excalibur  Precision  155mm  Projectiles  (Excalibur)  65 

F-22  Increment  3.2B  67 

F-35  Lightning  II  Program  (F-35)  69 

Family  of  Beyond  Line-of-Sight  -  Terminals  (FAB-T)  71 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78)  73 

Global  Positioning  System  III  (GPS  III)  75 

Ground/Air  Task  Oriented  Radar  (G/ATOR)  77 

Integrated  Air  and  Missile  Defense  (IAMD)  79 

Joint  Air-to-Surface  Standoff  Missile  Extended  Range 

(JASSM-ER)  81 

Joint  Light  Tactical  Vehicle  (JLTV)  83 

Joint  Precision  Approach  and  Landing  System  Increment  1A 

(JPALS  Inc  1  A)  85 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form 
Fit  Radios  (JTRS  HMS)  87 

KC-46  Tanker  Modernization  Program  (KC-46)  89 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6)  91 

Littoral  Combat  Ship  (LCS)  93 

Littoral  Combat  Ship  -  Mission  Packages  (LCS  MP)  95 

Mobile  User  Objective  System  (MUOS)  97 

MG-1C  Gray  Eagle  Unmanned  Aircraft  System 

(MO-1  C  Gray  Eagle)  99 


Page  i 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Contents 


MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton)  101 

MQ-8  (Fire  Scout)  103 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper)  105 

Next  Generation  Operational  Control  System  (GPS  OCX)  107 

P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (P-8A)  109 

Paladin  Integrated  Management  (PIM)  111 

Presidential  Helicopter  Fleet  Replacement  Program  (VXX)  113 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System 

(RQ-4A/B  Global  Hawk)  115 

Ship  to  Shore  Connector  Amphibious  Craft  (SSC)  1 1 7 

Small  Diameter  Bomb  Increment  II  (SDB  II)  119 

Space  Fence  121 

Warfighter  Information  Network-Tactical  Increment  2 

(WIN-T  Inc  2)  123 

Warfighter  Information  Network-Tactical  Increment  3 

(WIN-T  Inc  3)  125 

One-Page  Assessments  of  Individual  Programs 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio 

System  (AMF  JTRS)  127 

Amphibious  Combat  Vehicle  (ACV)  128 

Armored  Multi-Purpose  Vehicle  (AMPV)  129 

B-2  Defensive  Management  System  Modernization 

(B-2DMS)  130 

Combat  Rescue  Helicopter  (CRH)  131 

Common  Infrared  Countermeasure  (CIRCM)  132 

DDG  51  Destroyer  (DDG  51)  133 

F-15  Eagle  Passive/Active  Warning  and  Survivability  System 

(F-15  EPAWSS)  134 

Ground  Combat  Vehicle  (GCV)  135 

Indirect  Fire  Protection  Capability-Increment  2  (IFPC  Inc  2)  136 

Joint  Air-to-Ground  Missile  (JAGM)  137 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor 
System  (JLENS)  138 

Military  GPS  User  Equipment  (MGUE),  Increment  1  139 

Next  Generation  Jammer  (NGJ)  140 

Ohio-Class  Replacement  (OR)  141 

Presidential  Aircraft  Recapitalization  (PAR)  142 

Space  Based  Infrared  Satellite  System  (SBIRS  High)  143 

Three-Dimensional  Expeditionary  Long-Range  Radar 

(3DELRR)  144 

Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike 

(UCLASS)  System  145 


Page  ii 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Contents 


Agency  Comments  and  Our  Evaluation  146 


Appendixes 

Appendix  I:  Scope  and  Methodology  149 

Appendix  II:  Current  and  First  Full  Estimates  for  DOD’s  2013  Portfolio  of 

Major  Defense  Acquisition  Programs  160 

Appendix  III:  Changes  in  DOD’s  2013  Portfolio  of  Major  Defense  Acquisition 

Programs  over  5  Years  and  Since  First  Full  Estimates  164 

Appendix  IV:  Knowledge-Based  Acquisition  Practices  165 

Appendix  V:  Technology  Readiness  Levels  167 

Appendix  VI:  Major  Defense  Acquisition  Programs’  Individual 

Subcontracting  Reports  in  the  Electronic  Subcontracting 
Reporting  System  169 

Appendix  VII:  Comments  from  the  Department  of  Defense  172 

Appendix  VIII:  GAO  Contact  and  Staff  Acknowledgments  173 


Related  GAO  Products 


176 


Tables 


Table  1:  Changes  in  DOD’s  Portfolio  of  Major  Defense  Acquisition 
Programs  from  2012  to  2013 

Table  2:  Changes  in  DOD’s  2013  Portfolio  of  80  Major  Defense 
Acquisition  Programs  over  the  Past  Year 

Table  3:  Increases  in  Buying  Power  for  the  2013  Portfolio  over  the 
Past  Year 

Table  4:  Cost  Changes  in  DOD’s  10  Costliest  Programs  in 
Comparison  to  the  Rest  of  the  Current  Portfolio 

Table  5:  Cost  and  Schedule  Change  since  First  Full  Estimates  by 
System  Type 

Table  6:  Future  Funding  and  Future  Funding  Required  Due  to  Cost 
Growth  from  First  Full  Estimates 

Table  7:  Use  of  Activities  to  Ensure  Competition  Reported  by 
Future  and  Current  Programs 

Table  8:  Current  Cost  Estimates  and  First  Full  Estimates  for  DOD’s 
2013  Portfolio  of  Major  Defense  Acquisition 
Programs 

Table  9:  Cost  and  Schedule  Changes  for  Programs  in  DOD’s  2013 
Portfolio 


7 

10 

13 

18 

19 

21 

44 

160 

164 


Page  iii 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Contents 


Table  10: Best  Practices  for  Knowledge-based  Acquisitions 

Table  11:  Major  Defense  Acquisition  Programs’  Individual 

Subcontracting  Reports  in  the  Electronic  Subcontracting 
Reporting  System 

165 

170 

Figures 

Figure  1: 

DOD  Portfolio  Cost  and  Size,  2004-2013 

8 

Figure  2: 

Distribution  of  the  Total  Acquisition  Cost  Change  for  the 

2013  Portfolio 

11 

Figure  3: 

Comparison  of  the  Cost  Performance  of  DOD's  2011 , 

2012,  and  2013  Portfolios 

15 

Figure  4: 

Proportion  of  Development  and  Procurement  Funding  for 

the  10  Costliest  Programs 

17 

Figure  5: 

DOD's  Acquisition  Cycle  and  GAO  Knowledge  Points 

23 

Figure  6: 

Implementation  of  Knowledge-Based  Practices  for 

Programs  in  System  Development 

27 

Figure  7: 

Implementation  of  Knowledge-Based  Practices  for 

Programs  at  Critical  Design  Review 

31 

Figure  8: 

Implementation  of  Knowledge-Based  Practices  for 

Programs  at  Production  Decision 

34 

Figure  9: 

Realized  and  Expected  “Should-Cost”  Savings  Reported 

by  31  Programs 

41 

Figure  1C 

):  Illustration  of  Program  Two-Page  Assessment 

49 

Figure  11 

:  Examples  of  Knowledge  Scorecards 

51 

Page  iv 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Contents 


Abbreviations 

AEHF 

Advanced  Extremely  High  Frequency 

AMDR 

Air  and  Missile  Defense  Radar 

BMDS 

Ballistic  Missile  Defense  System 

CSB 

configuration  steering  board 

DAMIR 

Defense  Acquisition  Management  Information  Retrieval 

DDG  51 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer 

DOD 

Department  of  Defense 

eSRS 

Electronic  Subcontracting  Reporting  System 

G/ATOR 

Ground/Air  Task  Oriented  Radar 

GCV 

Ground  Combat  Vehicle 

JLTV 

Joint  Light  Tactical  Vehicle 

LCS 

Littoral  Combat  Ship 

MRL 

manufacturing  readiness  level 

NA 

not  applicable 

OMB 

Office  of  Management  and  Budget 

PIM 

Paladin  Integrated  Management 

SAR 

Selected  Acquisition  Report 

SIBRS 

Space  Based  Infrared  System 

TBD 

to  be  determined 

TRL 

technology  readiness  level 

WIN-T  Inc  3 

Warfighter  Integration  Network-Tactical  Increment  3 

WSARA 

Weapon  Systems  Acquisition  Reform  Act 

This  is  a  work  of  the  U.S.  government  and  is  not  subject  to  copyright  protection  in  the 
United  States.  The  published  product  may  be  reproduced  and  distributed  in  its  entirety 
without  further  permission  from  GAO.  However,  because  this  work  may  contain 
copyrighted  images  or  other  material,  permission  from  the  copyright  holder  may  be 
necessary  if  you  wish  to  reproduce  this  material  separately. 

Page  v 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Contents 


Page  vi 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


Comptroller  General 
of  the  United  States 


441  G  St.  N.W. 
Washington,  D.C.  20548 


March  31, 2014 
Congressional  Committees 

I  am  pleased  to  present  GAO’s  12th  annual  assessment  of  the  Department 
of  Defense’s  (DOD)  major  defense  acquisition  programs.  This  report  offers 
observations  on  the  performance  of  DOD’s  current  $1.5  trillion  portfolio  of 
80  programs.  At  its  current  size,  this  portfolio  will  require  almost  one-third 
of  all  expected  development  and  procurement  funding  during  the  next  5 
years.  With  the  decline  in  the  availability  of  discretionary  funding,  DOD 
cannot  afford  to  miss  opportunities  to  address  inefficiencies  in  these 
programs  to  free  up  resources  for  additional  high  priority  needs  such  as 
support  of  current  operations.  Consistent  with  our  past  recommendations, 
Congress  and  DOD  have  taken  meaningful  steps  over  the  past  several 
years  to  address  long-standing  problems  with  DOD  weapon  system 
acquisitions — an  area  that  has  been  on  GAO’s  high-risk  list  for  more  than 
20  years.1  We  are  encouraged  about  these  steps,  but  we  are  also  mindful 
that  previous  attempts  to  implement  best  practices  and  reign  in  cost  and 
schedule  overruns  have  failed  or  resulted  in  only  marginal  improvements. 

Our  current  assessment  shows  that  the  estimated  cost  of  DOD’s  2013 
portfolio  of  major  defense  acquisition  programs  is  $14.1  billion  more  than 
the  2012  portfolio  while  the  size  of  the  portfolio  decreased  from  85  to  80 
programs.2  This  places  the  size  and  cost  of  the  portfolio  on  par  with  what  it 
was  for  the  2004  portfolio,  as  opposed  to  the  peak  in  size  and  cost  seen  in 
the  $1.7  trillion,  97-program  portfolio  in  2010.  Over  the  past  year,  the  80 
programs  of  the  2013  portfolio  have  grown  by  a  total  of  $12.6  billion,  a  net 
cost  increase  nearly  all  attributable  to  the  significant  procurement  cost 
associated  with  one  program — the  Evolved  Expendable  Launch  Vehicle. 
The  cost  growth  of  the  portfolio  as  a  whole  is  not  necessarily  indicative  of 
the  performance  of  every  program  as  50  of  the  80  programs  reduced  their 


^AO,  High-Risk  Series:  An  Update,  GAO-13-283  (Washington,  D.C.:  February  2013). 

2Our  assessment  of  DOD’s  2012  portfolio  of  86  major  defense  acquisition  programs 
included  the  Missile  Defense  Agency’s  Ballistic  Missile  Defense  System  (BMDS).  While 
DOD  continues  to  report  on  BMDS  as  a  major  defense  acquisition  program,  we  exclude  it 
from  our  assessment  this  year  as  the  program  lacks  an  acquisition  program  baseline 
comparable  to  the  other  programs,  preventing  us  from  measuring  cost  and  schedule 
change  in  a  manner  consistent  with  our  methodology.  For  more  information  on  cost  and 
schedule  baselines  for  BMDS  and  the  challenges  for  oversight  they  present  see  GAO, 
Missile  Defense:  Opportunity  to  Refocus  on  Strengthening  Acquisition  Management,  GAO- 
13-432  (Washington,  D.C. :  Apr.  26,  2013). 
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costs  over  the  past  year.  While  a  large  number  of  programs  did  reduce 
cost  over  the  past  year,  the  need  for  continued  oversight  and  cost 
management  remains  as  many  programs  are  still  not  achieving  the  long- 
range  targets  for  limited  total  acquisition  cost  growth  discussed  by  GAO, 
DOD,  and  the  Office  of  Management  and  Budget.  Our  assessment  also 
shows  that  the  implementation  of  knowledge-based  acquisition  practices 
varied  across  the  portfolio,  though  programs  that  entered  over  the  past  5 
years  were  more  likely  to  follow  knowledge-based  principles  at 
development  start  thereby  making  them  more  likely  to  achieve  their  cost, 
schedule,  and  performance  objectives.  Finally,  our  analysis  shows  that 
many  programs  are  implementing  selected  acquisition  reforms  focused  on 
affordability  and  cost  savings,  although  DOD  continues  to  accept  risks  by 
allowing  programs  to  begin  production  before  completing  developmental 
testing.  This  report  also  includes  brief  assessments  of  56  future  and 
current  major  defense  programs  that  provide  additional  insights  into  the 
performance  of  the  portfolio. 

Current  and  anticipated  fiscal  constraints  make  it  imperative  that  DOD 
continue  to  find  ways  to  improve  the  efficiency  of  its  major  weapon 
systems  portfolio  while  still  delivering  the  capabilities  required  by  the 
warfighter.  While  many  of  the  recent  efficiencies  achieved  by  programs 
decreased  their  costs  without  reducing  quantities,  it  is  unclear  how  much 
more  savings  can  be  obtained  in  this  manner  or  if  the  estimates  of  future 
saving  based  upon  ongoing  DOD  efficiency  initiatives  will  come  to  fruition. 
Continued  strong  leadership  on  the  part  of  DOD  is  essential,  whether  on 
advancing  gains  made  through  “Better  Buying  Power”  initiatives  or  in 
enforcing  a  broader  implementation  of  best  practices  in  all  aspects  of  a 
new  weapon  system  acquisition  as  it  enters  the  portfolio. 


Gene  L.  Dodaro 
Comptroller  General 
of  the  United  States 
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U.S.  GOVERNMENT  ACCOUNTABILITY  OFFICE 


441  G  St.  N.W. 
Washington,  D.C.  20548 


March  31, 2014 
Congressional  Committees 

In  response  to  the  mandate  in  the  joint  explanatory  statement  to  the 
Department  of  Defense  Appropriations  Act,  2009,  this  report  provides 
perspectives  on  how  well  the  department  is  planning  and  executing  its 
$1 .5  trillion  portfolio  of  major  weapon  programs,  in  addition,  this  report 
includes  information  related  to  small  business  participation  pursuant  to  a 
mandate  in  a  report  accompanying  the  National  Defense  Authorization  Act 
for  Fiscal  Year  201 3. 1  The  current  portfolio’s  cost  has  declined  since 
peaking  at  $1.7  trillion  in  2010  and  is  currently  at  its  lowest  point  in  almost 
10  years.  In  addition,  the  number  of  programs  in  the  portfolio  has 
decreased  from  97  in  2010  to  80  programs  in  2013.  Since  the  first 
issuance  of  this  series  of  reports  in  2003,  Congress  and  DOD  have  made 
meaningful  improvements  to  the  statutory  and  policy  frameworks  that 
govern  the  defense  acquisition  system  by  mandating  and  encouraging  a 
more  knowledge-based  approach  for  major  weapon  programs.  However, 
we  have  noted  in  the  past  that  practice  has  lagged  behind  policy  in  certain 
areas  and  commensurate  improvements  in  program  outcomes  are  often 
not  evident  as  many  programs  continue  to  cost  more  and  take  longer  to 
develop  than  originally  estimated.  The  changes  in  DOD’s  portfolio  over  the 
past  few  years  indicate  that  some  improvements  are  being  realized  in  the 
near  term.  With  the  prospect  of  decreased  defense  budgets  in  the  near 
future,  it  is  imperative  that  DOD  continue  to  find  ways  to  reduce  cost  and 
improve  the  economy  and  efficiency  of  its  acquisition  practices  not  only  in 
the  short  term,  but  throughout  the  life  of  current  and  future  defense 
acquisition  programs.  Such  cost  reductions  may  be  found  by  changing 
program  acquisition  strategies  to  include  a  more  knowledge-based 
approach  through  the  implementation  of  acquisition  reforms  and  initiatives 
and  the  reduction  of  practices  like  concurrent  testing  and  production,  the 
risks  of  which  we  have  identified  repeatedly. 

This  report  includes  observations  on  (1)  the  cost  and  schedule 
performance  of  DOD’s  2013  portfolio  of  80  major  defense  acquisition 
programs,  (2)  the  knowledge  attained  at  key  junctures  in  the  acquisition 


^ee  Explanatory  Statement,  154  Cong.  Rec.  H  9427,  9526  (daily  ed.,  Sept.  24,  2008),  to 
the  Department  of  Defense  Appropriations  Act,  2009,  contained  in  Division  C  of  the 
Consolidated  Security,  Disaster  Assistance,  and  Continuing  Appropriations  Act,  2009,  Pub. 
L.  No.  110-329  (2008)  and  H  R.  Rep.  No.  112-479,  at  284  (2012),  accompanying  Pub.  L. 
No. 112-239. 
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process  for  38  weapon  programs  in  development  or  early  production,  and 
(3)  key  acquisition  reform  initiatives  and  whether  programs  are  conducting 
or  planning  concurrent  testing  and  production.2 

Our  observations  in  this  report  are  based  on  three  sets  of  programs: 

•  We  assessed  80  major  defense  acquisition  programs  in  DOD’s  2013 
portfolio  for  our  analysis  of  cost  and  schedule  performance.  We 
obtained  cost,  schedule,  and  quantity  data  from  DOD’s  December  2012 
Selected  Acquisition  Reports  (SAR)  and  from  the  Defense  Acquisition 
Management  Information  Retrieval  Purview  system. 

•  We  assessed  38  major  defense  acquisition  programs  that  were  mostly 
between  the  start  of  development  and  the  early  stages  of  production  for 
our  analysis  of  knowledge  attained  at  key  junctures  and  the 
implementation  of  acquisition  reforms.  We  obtained  information  on  the 
extent  to  which  the  programs  follow  knowledge-based  practices  for 
technology  maturity,  design  stability,  and  production  maturity  using  a 
data-collection  instrument.  We  also  submitted  a  questionnaire  to 
program  offices  to  collect  information  on  issues  such  as  systems 
engineering  reviews,  design  stability,  manufacturing  planning  and 
execution,  and  the  implementation  of  specific  acquisition  reforms.  We 
received  questionnaire  responses  from  all  38  programs  from  August  to 
November  2013. 

•  In  addition,  we  assessed  18  future  major  defense  acquisition  programs 
in  order  to  gain  additional  insights  into  knowledge  attained  before  the 
start  of  development  and  the  implementation  of  key  acquisition  reform 
initiatives.  We  submitted  a  questionnaire  to  program  offices  to  collect 
information  on  issues  such  as  program  schedule  events,  costs,  and 
numerous  acquisition  reforms,  and  received  responses  from  all  18 
programs  from  August  to  November  2013. 


2Major  defense  acquisition  programs  (MDAP)  are  those  identified  by  DOD  with  a  dollar 
value  for  all  increments  estimated  to  require  eventual  total  expenditure  for  research, 
development,  test,  and  evaluation  of  more  than  $480  million,  or  for  procurement  of  more 
than  $2.79  billion,  in  fiscal  year  2014  constant  dollars.  DOD  has  a  list  of  programs 
designated  as  future  major  defense  acquisition  programs.  These  programs  have  not 
formally  been  designated  as  MDAPs;  however,  DOD  plans  for  these  programs  to  enter 
system  development,  or  bypass  development  and  begin  production,  at  which  point  they  will 
likely  be  designated  as  MDAPs.  We  refer  to  these  programs  as  future  major  defense 
acquisition  programs  throughout  this  report. 
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In  addition  to  our  observations,  we  present  individual  assessments  of  56 
weapon  programs.  Selection  factors  include  major  defense  acquisition 
programs  in  development  or  early  production,  future  programs,  and 
recently  canceled  or  restructured  programs. 

We  conducted  this  performance  audit  from  June  2013  to  March  2014  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  based 
on  our  audit  objectives.  Appendix  I  contains  detailed  information  on  our 
scope  and  methodology. 


Observations  on  the 
Cost  and  Schedule 
Performance  of  DOD’s 
2013  Major  Defense 
Acquisition  Program 
Portfolio 


The  estimated  cost  of  DOD’s  2013  portfolio  of  80  weapon  programs  is 
$14.1  billion  more  than  the  2012  portfolio  of  85  programs.  The  80 
programs  within  this  year’s  portfolio  have  had  estimated  cost  increases  of 
$12.6  billion  against  their  estimates  from  a  year  ago  and  an  average  of  2 
months  increase  in  schedule.  This  increase  masks  otherwise  positive 
trends  as  50  of  80  programs  in  the  portfolio  decreased  their  total 
acquisition  costs  and  the  majority  of  the  cost  increases  can  be  traced  to 
either  the  effects  of  additional  procurement  quantities  or  inefficiencies 
experienced  in  a  few  programs.  Though  notable,  the  cost  reductions 
achieved  by  the  programs  in  the  201 3  portfolio  over  the  past  year  need  to 
be  weighed  against  the  increases  many  of  these  programs  have  had  over 
the  past  five  years  or  since  their  first  full  estimate  of  costs.  Our  analysis  of 
DOD’s  2013  portfolio  allows  us  to  make  the  following  eight  observations. 
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Cost  and  Schedule  Performance  Observations 

General  changes  in  the  portfolio 

1.  When  compared  to  the  2012  portfolio,  the  size  of  DOD’s  2013  portfolio  decreased 
from  85  to  80  programs,  but  its  overall  cost  has  increased  by  $14.1  billion  from 
$1,501  billion  to  $1,515  billion.  The  decrease  in  portfolio  size  follows  a  trend  from  the 
past  3  years.3 

2.  When  comparing  the  cost  and  schedule  estimates  of  the  80  programs  in  the  2013 
portfolio  we  found  cost  growth  of  $12.6  billion  and  an  average  schedule  delay  of  2 
months  over  the  past  year.  When  assessed  against  first  full  estimates,  the  total  cost 
of  the  2013  portfolio  has  increased  by  nearly  $448  billion  with  an  average  delay  of  28 
months  in  operating  capability.11 

Factors  that  explain  the  changes 

3.  While  the  overall  cost  of  the  2013  portfolio  has  increased,  50  of  the  80  programs 
within  the  portfolio  reduced  their  costs  over  the  past  year.  The  majority  of  the  net  cost 
growth  can  be  attributed  to  a  single  program,  the  Evolved  Expendable  Launch 
Vehicle  (EELV). 

4.  Our  analysis  shows  that,  when  the  effects  of  quantity  changes  are  accounted  for, 
DOD  improved  buying  power  on  64  percent  of  the  programs  in  the  portfolio  over  the 
past  year. 

5.  As  measured  against  metrics  discussed  by  GAO,  the  Office  of  Management  and 
Budget,  and  DOD  in  2008,  only  55  percent  of  programs  in  the  201 3  portfolio  meet  the 
metric  for  less  than  10  percent  growth  over  the  past  5  years  and  only  44  percent 
meet  the  metric  for  less  than  1 5  percent  growth  since  first  full  estimates;  both 
proportions  are  smaller  than  in  the  2012  portfolio,  which  demonstrates  that  a 
program  can  experience  an  interval  of  cost  stability  between  periodic  cost  increases. 

Other  observations 

6.  The  total  acquisition  cost  of  the  portfolio  is  driven  in  large  part  by  the  10  costliest 
programs,  which  represent  59  percent  of  current  portfolio’s  total  cost.  These  10 
programs  alone  incurred  a  net  cost  increase  over  the  past  year  of  approximately 
$15.4  billion  due  primarily  to  quantity  increases  on  the  EELV,  while  the  other  70 
programs  reported  a  net  cost  decrease  of  $2.8  billion  over  the  same  period. 

7.  The  majority  of  the  cost  of  the  portfolio  is  concentrated  in  two  system  types  and  the 
amount  and  type  of  cost  growth  from  first  full  estimate,  as  well  as  the  schedule  delay, 
varied  significantly  by  system  type. 

8.  DOD  has  already  been  appropriated  more  than  $833  billion  for  the  current  portfolio, 
leaving  approximately  $682  billion  in  costs  to  be  funded,  mostly  for  procurement. 
Approximately  45  percent  of  the  remaining  cost  represents  growth  from  first  full 
estimates.  Almost  two-thirds  of  the  future  funding  is  needed  for  1 0  programs,  with  35 
percent  of  the  total  costs  remaining  required  for  the  F-35  program  alone. 


aAII  dollar  figures  are  in  fiscal  year  2014  constant  dollars,  unless  otherwise  noted. 

bTotal  acquisition  cost  includes  acquisition-related  operation  and  maintenance  costs  and  system- 
specific  military  construction  costs  in  addition  to  development  and  procurement  costs. 


Additional  details  about  each  observation  follow. 
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1.  When  compared  to  the  2012  portfolio,  the  size  of  DOD’s  2013 
portfolio  decreased  from  85  to  80  programs,  but  its  overall  cost 
has  increased  by  $14.1  billion,  from  $1,501  billion  to  $1,515 
billion.  The  decrease  in  portfolio  size  follows  a  trend  from  the 
past  3  years.  With  80  programs  and  an  estimated  total  cost  of  more 
than  $1 .5  trillion,  the  2013  portfolio  has  fewer  programs  but  a  larger 
total  acquisition  cost  when  compared  to  the  2012  portfolio.  The 
changes  from  the  2012  portfolio  to  the  2013  portfolio  are  outlined  in 
table  1  below. 


Table  1:  Changes  in  DOD’s  Portfolio  of  Major  Defense  Acquisition  Programs  from 

2012  to  2013 

Fiscal  year  2014  dollars  in  billions 

2012  portfolio  (85  programs) 

$1,501.1 

Less  cost  of  12  exiting  programs 

-$73.1 

Plus  estimated  total  cost  of  7  entering  programs 

+$101.6 

Less  net  cost  changes  of  73  remaining  programs 

-$14.4 

2013  portfolio  (80  programs) 

$1,515.2 

Source:  GAO  analysis  of  DOD  data. 

Note:  Our  assessment  of  DOD’s  2012  portfolio  included  the  Ballistic  Missile  Defense  System  (BMDS). 
We  exclude  it  from  our  assessment  this  year  as  the  program  lacks  an  acquisition  program  baseline 
comparable  to  other  programs,  preventing  us  from  measuring  cost  and  schedule  change  in  a  manner 
consistent  with  our  methodology.  To  make  the  comparisons  above  we  adjusted  the  2012  portfolio 
estimates  to  reflect  the  removal  of  BMDS.  Some  numbers  may  not  sum  due  to  rounding. 

The  2013  portfolio  reflects  a  net  reduction  of  5  programs — from  85  to 
80 — and,  when  all  the  programmatic  changes  are  taken  into  account, 
the  cost  of  the  2013  portfolio  is  $14.1  billion  more  than  the  2012 
portfolio.3  The  current  size  of  the  portfolio  in  terms  of  number  of 


3The  12  programs  that  exited  the  portfolio  were  AIM-9X  Block  I,  C-130  Avionics 
Modernization  Program,  Chemical  Demilitarization-Chemical  Materials  Agency,  Cobra 
Judy  Replacement,  High  Mobility  Artillery  Rocket  System,  Joint  High  Speed  Vessel,  Joint 
Tactical  Radio  System  Ground  Mobile  Radio,  Multi-Platform-Radar  Technology  Insertion 
Program,  National  Polar-orbiting  Operational  Environmental  Satellite  System,  Stryker 
Family  of  Vehicles,  Thermal  Weapons  Sight,  Warfighter  Information  Network  -  Tactical 
Increment  1.  The  seven  that  entered  the  portfolio  were  Airborne  Warning  and  Control 
System  Block  40/45  Upgrade,  AN/TPS-80  Ground/Air  Task  Oriented  Radar,  B61  Mod  12 
Life  Extension  Program  Tailkit  Assembly,  Evolved  Expendable  Launch  Vehicle,  Joint  Light 
Tactical  Vehicle,  Next  Generation  Operational  Control  System  (GPS  OCX),  and  Ship  to 
Shore  Connector. 
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programs  and  total  cost  is  similar  to  that  seen  in  the  2004  portfolio  and 
furthers  a  trend  for  decreased  portfolio  size,  and  to  some  extent  cost, 
seen  since  the  2010  portfolio.  Figure  1  shows  the  cost  and  size  of 
DOD’s  portfolios  of  major  weapons  acquisitions  over  the  past  1 0  years. 


Figure  1:  DOD  Portfolio  Costand  Size,  2004-2013 

Fiscal  year  2014  dollars  (in  billions) 

1,700 


Number  of  programs 
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Total  portfolio  acquisiton  cost 

•  Number  of  programs 
Source:  GAO  analysis  of  DOD  data. 

Note:  The  2009  portfolio  is  excluded  because  there  were  no  annual  SARs  released  for  the  December 
2008  submission  date. 

A  closer  look  at  the  7  programs  that  entered  the  portfolio  shows  that  4 
entered  the  portfolio  at  the  beginning  of  system  development.  In 
contrast,  two  of  the  other  three  programs — the  AN/TPS-80  Ground/Air 
Task  Oriented  Radar  (G/ATOR)  and  the  Airborne  Warning  and  Control 
System  Block  40/45  Upgrade — entered  the  portfolio  long  after  their 
development  start  and  are  in  production  or  will  have  a  production 
decision  in  2013.  These  programs  were  not  originally  planned  as  major 
acquisitions,  but  their  costs  grew  past  the  threshold  for  major  defense 
acquisition  programs.  For  example,  G/ATOR  began  system 
development  in  2005  as  a  non-major  defense  acquisition  program  and 
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was  reclassified  due  to  program  cost  growth  realized  after  completing 
6  years  of  an  8-year  development  effort.  The  final  program,  EELV,  re¬ 
entered  the  portfolio  in  response  to  statutory  direction  and  soon  after 
reported  a  quantity  increase  and  a  critical  breach  of  Nunn-McCurdy 
unit  cost  threshold.4 

Of  the  12  programs  that  exited  the  portfolio,  8  completed  development 
and  fielded  a  system,  while  three  were  canceled  after  reporting 
substantial  development  cost  growth  or  delays  to  their  planned  initial 
capability  or,  in  one  case,  both.  A  final  program,  the  Joint  High  Speed 
Vessel,  exited  the  portfolio  after  a  quantity  reduction  caused  its  cost  to 
slip  below  the  threshold  for  major  defense  acquisition  programs. 

2.  When  comparing  the  cost  and  schedule  estimates  of  the  80 
programs  in  the  2013  portfolio  we  found  cost  growth  of  $12.6 
billion  and  average  schedule  delay  of  2  months  over  the  past 
year.  While  the  above  observation  states  the  change  from  the  85 
program  2012  portfolio  due  to  programs  exiting  and  entering,  in  this 
observation  we  measure  the  change  that  occurred  only  on  the  80 
programs  in  the  2013  portfolio.  Table  2  shows  the  changes  in  cost  and 
schedule  for  the  2013  portfolio  in  the  past  year. 


4Section  2433  of  title  10  of  the  United  States  Code,  commonly  referred  to  as  Nunn- 
McCurdy,  requires  DOD  to  notify  Congress  whenever  a  major  defense  acquisition 
program’s  unit  cost  experiences  cost  growth  that  exceeds  certain  thresholds.  This  is 
commonly  referred  to  as  a  Nunn-McCurdy  breach.  Significant  breaches  occur  when  the 
program  acquisition  unit  cost  or  procurement  unit  cost  increases  by  at  least  1 5  percent  over 
the  current  baseline  estimate  or  at  least  30  percent  over  the  original  estimate.  For  critical 
breaches,  when  these  unit  costs  increase  at  least  25  percent  over  the  current  baseline 
estimate  or  at  least  50  percent  over  the  original,  DOD  is  required  to  take  additional  steps, 
including  conducting  an  in-depth  review  of  the  program.  Programs  with  critical  breaches 
must  be  terminated  unless  the  Secretary  of  Defense  certifies  to  certain  facts  related  to  the 
program  and  takes  other  actions,  including  restructuring  the  program.  10  U.S.C.  §  2433a. 
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Table  2:  Changes  in  DOD’s  2013  Portfolio  of  80  Major  Defense  Acquisition  Programs  over  the  Past  Year 


Fiscal  year  2014  dollars  in  billions 

Estimated  2013 
portfolio  cost  in  2012 

Estimated  2013 
portfolio  cost  in  2013 

Increase 
since  2012 

Percentage  change 
since  2012 

Total  estimated  research  and  development  cost 

$286.7 

$288.7 

$2.0 

0.7% 

Total  estimated  procurement  cost 

1,201.6 

1,213.2 

11.5 

1.0 

Total  other  acquisition  costs3 

14.3 

13.3 

-0.9 

-6.3 

Total  estimated  acquisition  cost 

1,502.6 

1,515.2 

12.6 

0.8 

Average  delay  in  delivering  initial  capabilities 

— 

— 

2  months 

2.8% 

Source:  GAO  analysis  of  DOD  data. 

Note: 

Some  numbers  may  not  sum  due  to  rounding. 

aln  addition  to  research  and  development  and  procurement  costs,  total  acquisition  cost  includes 

acquisition-related  operation  and  maintenance  and  system-specific  military  construction  costs. 

The  rise  in  total  acquisition  cost  within  the  current  portfolio  over  the 
past  year  is  attributable  to  cost  growth  in  both  development  and 
procurement,  with  procurement  accounting  for  more  than  90  percent  of 
the  total  increase.  Decreases  in  other  costs,  such  as  military 
construction,  result  in  a  net  cost  growth  of  $12.6  billion  for  these  80 
programs  over  the  past  year.  When  measured  against  programs’  first 
full  estimates,  the  total  acquisition  cost  of  the  current  portfolio  has 
grown  by  $447.8,  or  nearly  42  percent.  The  average  delay  in  delivering 
initial  capabilities  has  slipped  an  additional  2  months  over  the  past 
year.  The  average  delay  against  first  full  estimates  is  28  months,  on 
average. 

3.  While  the  overall  cost  of  the  2013  portfolio  has  increased,  50  of 
the  80  programs  within  the  portfolio  reduced  their  costs  over  the 
past  year.  The  majority  of  the  net  cost  growth  can  be  attributed  to 
a  single  program — EELV.  The  $12.6  billion  increase  shown  in  table  2 
above  is  the  net  result  of  cost  changes  on  all  80  programs  in  the 
current  portfolio.  The  distribution  of  those  cost  changes  across  the 
portfolio  is  shown  below  in  figure  2. 
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Figure  2:  Distribution  of  the  Total  Acquisition  Cost  Change  for  the  2013  Portfolio 

Fiscal  year  2014  dollars  (in  billions) 


Amount  of  cost  change  in  each  percentage  change  interval  over  the  past  year 

Number  of  programs  in  each  percentage  change  interval 
Source:  GAO  analysis  of  DOD  data. 

Our  analysis,  as  represented  above,  shows  that  50  programs 
experienced  cost  decreases  totaling  almost  $30.9  billion  during  the 
past  year  while  30  programs  reported  cost  increases  totaling 
approximately  $43.5  billion.  Of  the  programs  that  decreased  their  total 
acquisition  costs, 

•  38  did  so  by  finding  efficiencies  within  the  program  and  not  by 
decreasing  procurement  quantities; 

•  9  reduced  their  procurement  quantities  thereby  reducing  overall 
cost;  and 

•  3  increased  their  procurement  quantities  but  found  enough  offsetting 
efficiencies  to  reduce  overall  cost. 

The  most  significant  of  these  decreases  is  the  $11.5  billion  reduction  to 
the  F-35  Joint  Strike  Fighter  program’s  estimate,  due  solely  to 
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efficiencies  found  within  the  program  as  no  decrease  in  quantities  was 
reported.  In  contrast,  30  programs  reported  cost  increases.  Of  these, 

•  17  reported  cost  increases  due  to  inefficiencies  in  the  program  and 
not  any  change  to  procurement  quantities; 

•  12  programs  increased  their  procurement  quantities;  and 

•  1  program  reduced  estimated  procurement  quantities  but  still 
reported  an  increase  to  total  costs. 

The  most  significant  of  the  programs  with  increases  to  total  acquisition 
cost  was  EELV,  which  reported  an  increase  in  the  number  of  launch 
services  it  expected  to  procure  and  a  $28.1  billion,  or  78  percent, 
increase  in  total  acquisition  cost  during  the  past  year.  As  can  be  seen 
from  the  preceding  tally  of  individual  cost  increases  and  decreases,  to 
better  understand  the  changes  to  cost  the  effect  of  changes  to  quantity 
must  be  understood. 

4.  Our  analysis  shows  that,  when  the  effects  of  quantity  changes  are 
accounted  for,  DOD  improved  buying  power  on  64  percent  of  the 
programs  in  the  portfolio  over  the  past  year.  Although  procurement 
costs  for  the  portfolio  increased  by  $11.5  billion  over  the  past  year,  this 
growth  is  lower  than  GAO’s  calculations  of  the  expected  cost  growth 
due  to  quantity  changes,  indicating  that  programs  found  efficiencies  in 
other  areas  and  increased  their  buying  power.5  In  general,  buying 
power  can  be  defined  as  the  amount  of  good  or  service  that  can  be 
purchased  given  a  specified  level  of  funding.  Our  calculation  of  how 
programs’  cost  and  quantity  changes  affected  their  buying  power  are 
presented  in  table  3. 


5A  description  of  this  calculation  can  be  found  in  the  detailed  scope  and  methodology  in 
appendix  I. 
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Table  3:  Increases  in  Buying  Power  for  the  2013  Portfolio  over  the  Past  Year 

Fiscal  year  2014  dollars  in  billions 

Number  of 
programs 

Actual 
procurement 
cost  change 

GAO-calculated 
cost  change 
attributable  to 
quantity 
changes 

GAO-calculated 
cost  change  not 
attributable  to 
quantity  changes 

Increased  buying  power 

51 

-$23.0 

$4.9 

-$27.9 

Procurement  cost  decreased  with  no  quantity  change 

35 

-22.6 

0 

-22.6 

Quantity  increased  with  less  cost  increase  than  anticipated 

10 

4.8 

7.9 

-3.1 

Quantity  decreased  with  more  cost  decrease  than  anticipated 

6 

-5.2 

-3.0 

-2.2 

Decreased  buying  power 

25 

$34.5 

$22.4 

$12.1 

Procurement  cost  increased  with  no  quantity  change 

16 

4.7 

0 

4.7 

Quantity  increased  with  more  cost  increase  than  anticipated 

5 

32.1 

25.3 

6.8 

Quantity  decreased  with  less  cost  decrease  than  anticipated 

4 

-2.3 

-2.9 

0.5 

No  change  in  buying  power 

4 

$0 

$0 

$0 

Total 

80 

$11.5 

$27.3 

-$15.8 

Source:  GAO  analysis  of  DOD  data. 


Note:  Some  numbers  may  not  sum  due  to  rounding. 

Based  on  our  analysis,  a  total  of  51  programs  increased  their  buying 
power  in  the  past  year,  resulting  in  a  total  of  $23  billion  in  procurement 
cost  reductions.  Thirty-five  of  the  programs  that  decreased  their 
procurement  cost  did  so  without  reducing  quantities,  an  indication  that 
they  found  efficiencies  elsewhere.  Ten  programs  increased  their 
expected  procurement  quantities  but  their  total  procurement  cost 
increase  was  less  than  would  be  expected  based  upon  the  average 
procurement  unit  cost  of  each  program  used  in  GAO’s  calculation.  For 
example,  over  the  past  year  the  DDG  51  program  added  two  ships  to 
its  planned  procurement,  at  an  estimated  unit  cost  of  more  than  $1 .3 
billion.  If  the  resulting  procurement  cost  increase  from  these  additional 
quantities  is  ignored  the  cost  for  the  program  actually  decreases,  due 
to  the  fact  that  it  found  efficiencies  elsewhere  that  increased  the 
program’s  buying  power  even  as  the  program  increased  quantities.  An 
additional  6  programs  decreased  their  quantities  but  achieved  cost 
reductions  in  excess  of  that  anticipated  by  GAO’s  cost  calculations. 
For  example,  the  Littoral  Combat  Ship  (LCS)  program  reported  a 
procurement  cost  decrease  of  over  $3  billion.  While  a  significant 
portion  of  this  reduction  was  achieved  by  removing  three  ships  from 
the  planned  procurement,  almost  half,  $1.2  billion,  was  due  to  factors 
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other  than  the  quantity  reduction,  such  as  a  revised  estimate  that 
reflects  lower  future  costs. 

Increases  in  buying  power  such  as  these  could  be  expected  to 
continue  as  many  of  the  programs  that  responded  to  our  questionnaire 
indicated  that  they  expected  large  amounts  of  “should-cost”  savings  in 
the  future.  The  “should-cost”  initiative,  introduced  through  DOD’s 
“Better  Buying  Power”  initiatives,  emphasizes  the  importance  of  driving 
cost  improvements  during  contract  negotiation  and  program  execution 
to  control  costs,  improvements  that  could  lead  to  program  efficiencies 
which  increase  buying  power.  For  example,  the  AIM-9X  Block  II  Air-to- 
Air  Missile  program  realized  a  procurement  cost  decrease  of 
approximately  $327  million  with  no  change  in  quantity  over  the  past 
year  and  also  reported  $128  million  in  “should-cost”  savings  that  are 
expected  in  the  future.  In  another  case,  the  MQ-4C  Triton  Unmanned 
Aircraft  System  also  realized  decreasing  procurement  costs  of  $274 
million  over  the  past  year  while  maintaining  the  same  quantity  and 
reported  that  another  $480  million  in  “should-cost”  savings  are 
expected  in  the  future. 

Our  analysis  in  table  3  above  also  shows  that  procurement  costs  for  25 
programs  increased  by  $34.5  billion,  but  by  our  calculation  only  $22.4 
billion  can  be  attributed  to  the  cost  of  procuring  additional  units,  with 
$12.1  billion  of  that  increase  attributable  to  other  factors.  The  most 
significant  increase  was  that  of  the  EELV  program,  which  reported 
increasing  its  expected  launches,  the  program’s  measure  of  quantity, 
by  60  resulting  in  an  estimated  $28  billion  procurement  cost  increase, 
$6  billion  of  which  was  not  attributable  to  the  quantity  increase. 

5.  As  measured  against  metrics  discussed  by  GAO,  the  Office  of 
Management  and  Budget  (OMB),  and  DOD  in  2008,  only  55 
percent  of  programs  in  the  2013  portfolio  meet  the  metric  for  less 
than  10  percent  cost  growth  over  5  years  and  44  percent  meet  the 
metric  for  less  than  15  percent  growth  since  full  first  estimates; 
both  proportions  are  smaller  than  in  the  2012  portfolio,  which 
demonstrates  that  a  program  can  experience  an  interval  of  cost 
stability  between  periodic  cost  increases.  In  December  2008,  DOD, 
OMB,  and  GAO,  discussed  a  set  of  outcome  metrics  and  goals  to 
measure  program  cost  performance  over  time.  The  metrics  are 
intended  to  measure  total  program  cost-performance  on  a  percentage 
basis  over  three  time  periods:  the  preceding  year,  the  preceding  5 
years,  and  the  period  since  first  full  program  estimates  were 
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established.  We  have  reported  on  these  outcomes  in  past 
assessments.  Seventy-six  percent  of  the  programs  in  the  2013 
portfolio  are  meeting  the  one-year  cost  performance  metric  by  limiting 
their  total  acquisition  cost  growth  to  less  than  2  percent,  consistent 
with  the  cost  decreases  and  increases  explained  earlier.  Fewer 
programs,  however,  are  meeting  the  metrics  for  limited  growth  in  the 
past  five  years  and  since  first  full  estimates;  55  percent  meet  the  5- 
year  metric  while  44  percent  are  meeting  the  since  first  full  estimate 
metric.  This  demonstrates  that  some  programs  with  reduced  or  stable 
costs  over  the  past  year  have  previously  had  one  or  more  cost 
increases.  Programs  do  not  generally  experience  cost  change  in  a 
linear  fashion  with  set  amounts  of  increase  or  decrease  every  year, 
rather  they  have  periods  of  more  dramatic  change  interspersed  with 
periods  of  stability.  The  performance  of  the  2013  portfolio,  as  well  as 
the  two  previous  weapon  system  portfolios,  against  these  metrics  is 
shown  in  figure  3. 


Figure  3:  Comparison  of  the  Cost  Performance  of  DOD's  2011, 2012,  and  2013 
Portfolios 
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Source:  GAO  analysis  of  DOD  data. 
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Note:  Cost  growth  metric  is  defined  as  the  percentage  change  in  total  program  acquisition  cost. 
Analysis  includes  the  evaluation  of  cost  growth  on  individual  programs  in  the  portfolio  for  each  year. 
For  programs  where  cost  growth  is  measured  on  separate  program  sub-element,  the  elements  were 
combined  to  calculate  the  cost  growth  on  the  entire  program  as  a  whole. 

As  a  whole,  aggregate  cost  performance  of  the  2013  portfolio  has 
degraded  across  all  three  metrics  from  that  observed  in  the  2012 
portfolio.  The  percentage  increase  from  2012  to  2013  in  programs  that 
do  not  meet  the  cost  metrics  is  attributable  to  two  factors.  First,  many 
of  the  1 2  programs  that  are  no  longer  in  the  portfolio  were  meeting  one 
or  more  of  the  three  cost  metrics  while  2  of  the  7  that  entered — EELV 
and  G/ATOR — do  not  meet  one  or  more  metrics.  Second,  performance 
against  the  cost  metrics  has  grown  worse  on  some  of  the  ongoing 
programs.  For  example,  the  number  of  programs  that  do  not  meet  the 
1 -year  cost  growth  metric  increased  by  5 — eight  programs  improved  to 
meet  this  criterion  in  2013  while  the  cost  performance  on  13  other 
programs  worsened  over  the  past  year.  Despite  decreases  in  this 
metric  in  the  past  year,  the  portfolio’s  performance  for  2013  is  an 
improvement  on  the  2011  portfolio,  the  first  year  we  used  this  metric. 

6.  The  10  highest-cost  programs  account  for  59  percent  of  the 
development  and  procurement  cost  and  the  majority  of  cost 
growth.  As  we  have  reported  in  past  assessments,  the  total 
acquisition  cost  of  the  portfolio  is  driven  in  large  part  by  the  1 0  costliest 
programs,  which  currently  represent  59  percent  of  the  portfolio’s 
development  and  procurement  cost  as  shown  in  figure  4  below. 
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Figure  4:  Proportion  of  Development  and  Procurement  Funding  for  the  10  Costliest 
Programs 


remaining  programs 

Source:  GAO  analysis  of  DOD  data. 


Interestingly,  6  of  the  10  costliest  programs  in  the  current  portfolio  were 
also  in  the  list  of  10  costliest  programs  in  the  2004  portfolio,  and  if  each 
of  these  programs  had  executed  according  to  the  baseline  approved  at 
development  start,  3  of  them  would  have  completed  acquisition  and 
exited  the  portfolio. 

Over  the  past  year,  the  10  costliest  programs  incurred  a  net  cost 
increase  of  approximately  $15.4  billion,  due  primarily  to  quantity 
increases  on  the  EELV  program.  In  contrast,  the  other  70  programs 
reported  a  combined  cost  decrease  of  $2.8  billion.  Similarly,  the  10 
costliest  programs  also  account  for  almost  60  percent  of  all  cost 
growth — or  $124.4  billion — over  the  past  5  years  in  the  current 
portfolio.  Table  4  identifies  the  1 0  costliest  programs  and  lists  their  total 
acquisition  cost  and  cost  growth  in  the  past  year. 
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Table  4:  Cost  Changes  in  DOD’s  10  Costliest  Programs  in  Comparison  to  the  Rest  of 
the  Current  Portfolio 


Fiscal  Year  2014  dollars  in  billions 

Program  name 

Current  total 
estimated 
acquisition  cost 

Change  in 
acquisition  cost 
over  the  past  year 

F-35  Joint  Strike  Fighter 

$332.3 

-$11.5 

DDG  51  Arteigh  Burke  Class  Destroyer 

107.8 

2.4 

Virginia  Class  Submarine 

84.4 

-2.3 

Evolved  Expendable  Launch  Vehicle 

63.9 

28.1 

V-22  Osprey 

60.7 

0.8 

F/A-18E/F  Super  Hornet 

59.7 

-1.0 

Trident  II  Missile 

55.5 

0.6 

KC-46  Tanker 

44.5 

-1.2 

CVN  78  Gerald  R.  Ford  Class  Aircraft  Carrier 

36.0 

-0.3 

P-8A  Poseidon 

34.3 

-0.04 

10  costliest  programs 

$880.0 

$15.4 

Total  for  rest  of  the  current  portfolio 

$635.2 

-$2.8 

2013  portfolio  total 

$1,515.2 

$12.6 

Source:  GAO  analysis  of  DOD  data. 

Note:  Some  numbers  may  not  sum  due  to  rounding. 

7.  The  majority  of  the  cost  of  the  portfolio  is  concentrated  in  two 
system  types  and  the  amount  and  type  of  cost  growth  from  first 
full  estimate,  as  well  as  the  schedule  delay,  varied  significantly  by 
system  type.  Over  60  percent  of  the  current  portfolio’s  cost  is 
accounted  for  by  two  system  types — fixed  wing  manned  aircraft  and 
shipbuilding  programs,  both  of  which  experienced  unfavorable  cost 
and  schedule  growth  from  first  full  estimates.  The  amount  of  cost 
growth,  and  whether  that  growth  occurred  in  development  or 
procurement,  varied  by  the  type  of  system  acquired.  Table  5  shows  the 
amount  of  cost  growth  for  development  and  acquisition  unit  cost 
experienced  by  each  system  type  as  well  as  the  amount  of  schedule 
growth. 
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Table  5:  Cost  and  Schedule  Change  since  First  Full  Estimates  by  System  Type 

System  type 

Total  development 
cost  percentage 
increase 

Procurement  unit 
cost  percentage 
increase 

Percentage 
increase  in  time  to 
deliver  initial 
capabilities 

Shipbuilding 

113% 

19% 

53% 

Unmanned  vehicle 

86 

22 

58 

Rotary  wing 

78 

74 

30 

Satellite 

55 

85 

93 

Fixed  wing 

43 

55 

34 

C4ISR 

42 

-8 

26 

Munition 

17 

-10 

16 

Missile 

13 

79 

30 

Ground 

5 

62 

19 

Source:  GAO  analysis  of  DOD  data. 


Note:  The  procurement  unit  cost  growth  estimate  was  calculated  using  a  weighted  average  based 
each  program’s  current  procurement  cost  that  gives  more  expensive  programs  a  greater  value.  Three 
of  the  80  programs  in  the  current  portfolio — Chemical  Demilitarization-Assembled  Chemical  Weapons 
Alternatives,  EELV,  and  Ship  to  Shore  Connector — were  not  assessed  here  as  they  do  not  correspond 
with  any  of  the  nine  system  types  used. 


The  development  cost  growth  for  shipbuilding  programs  is  the  greatest 
among  all  system  types  and,  at  approximately  113  percent,  is  more 
than  double  that  experienced  by  the  portfolio  as  a  whole.  For  example, 
the  DDG  1000  and  LCS  have  experienced  more  than  342  percent  and 
148  percent  growth,  respectively,  in  development  costs  from  first  full 
estimates.  Shipbuilding  programs  have  also  experienced  significant 
schedule  delays  with  four  out  of  seven  programs  reporting  delays  to 
the  delivery  of  initial  capability  of  2  years  or  more.  We  have  an 
extensive  body  of  work  that  covers  the  problems  DOD  has 
experienced  in  developing  these  systems.6 


6GAO,  Navy  Shipbuilding:  Significant  Investments  in  the  Littoral  Combat  Ship  Continue 
Amid  Substantial  Unknowns  about  Capabilities,  GAO-13-530  (Washington,  D.C.:  July  22, 
2013);  Ford-Class  Carriers:  Lead  Ship  Testing  and  Reliability  Shortfalls  Will  Limit  Initial 
Fleet  Capabilities,  GAO-13-396  (Washington,  D.C.:  Sept.  5,  2013);  Arleigh  Burke 
Destroyers:  Additional  Analysis  and  Oversight  Required  to  Support  the  Navy's  Future 
Surface  Combatant  Plans,  GAO-12-113  (Washington,  D.C.:  Jan.  24,  2012);  Best  Practices: 
High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial  Shipbuilding  from  Navy 
Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009);  and  Defense  Acquisitions: 
Cost  to  Deliver  Zumwalt-Class  Destroyers  Likely  to  Exceed  Budgets,  GAO-08-804 
(Washington,  D.C.:  July.  31,  2008). 
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Rotary  wing  systems  have  also  experienced  exceptionally  high  levels 
of  development  cost  change,  with  4  out  of  9  rotary  wing  programs 
experiencing  more  than  150  percent  growth.  Rotary  wing  programs 
have  also  experienced  high  amounts  of  growth  in  procurement  unit 
cost.  As  an  example,  both  the  H-1  Upgrades  and  CH-47F  Improved 
Cargo  Helicopter  have  each  experienced  more  than  150  percent 
growth  in  their  average  procurement  unit  cost  since  first  full  estimates. 
Satellite  programs  show  above  average  growth  in  development  cost, 
procurement  unit  cost,  and  schedule  growth  since  first  full  estimates 
when  compared  to  the  portfolio  as  a  whole.  For  example,  Wideband 
Global  SATCOM  and  Space  Based  Infrared  System  (SIBRS)  High, 
have  each  experienced  more  than  100  percent  growth  in  their 
development  costs  while  another  satellite  program,  the  Advanced 
Extremely  High  Frequency  (AEHF)  Satellite,  has  experienced  almost 
200  percent  growth  in  its  average  procurement  unit  cost.  Three 
satellite  programs  have  also  experienced  schedule  growth  well  above 
the  portfolio  average  of  36  percent  with  each  of  them  reporting  delays 
of  4  years  or  more.  Unmanned  vehicles  have  also  experienced  greater 
than  average  schedule  delays  with  three  of  six  unmanned  vehicle 
programs  reporting  delays  of  3  years  or  more. 

8.  DOD  has  already  been  appropriated  more  than  $833  billion  for  the 
current  portfolio,  leaving  approximately  $682  billion  in  costs  to  be 
funded,  mostly  for  procurement.  Approximately  45  percent  of  the 
remaining  cost  represents  growth  from  first  full  estimates. 
Almost  two-thirds  of  the  future  funding  is  needed  for  10 
programs,  with  35  percent  of  the  total  costs  remaining  required 
for  the  F-35  program  alone.  A  larger  share  of  the  current  portfolio’s 
cost  is  now  appropriated  than  had  been  appropriated  for  the  portfolios 
from  1 0  or  even  5  years  ago.  Fifty-five  percent  of  the  costs  for  the  201 3 
portfolio  have  already  been  appropriated,  more  than  $200  billion  more 
than  in  the  2004  portfolio.  Forty-five  percent  of  the  total  cost  of  the 
current  portfolio — or  $682  billion — has  not  yet  been  funded  and  almost 
94  percent  of  the  remaining  funding — or  $638  billion — is  needed  for 
procurement. 

Approximately  45  percent — or  $31 1 .6  billion — of  that  future  funding  is 
needed  to  cover  cost  growth  from  first  full  estimates  for  programs  in 
the  current  portfolio.  These  funds  represent  resources  that  will  not  be 
available  to  support  other  defense  or  government  priorities.  We 
recognize  that  for  some  programs  a  portion  of  this  cost  growth  from 
first  full  estimates  may  be  attributable  to  a  quantity  increase.  For 
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example,  the  $14  billion  in  future  funding  needed  for  the  DDG  51 
Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51)  is  a  result  of 
quantity  increases  and  the  program  has  realized  a  significant  decrease 
in  its  average  procurement  unit  cost  from  first  full  estimates.  However, 
in  our  assessments  since  2011  we  have  found  that  a  substantial 
amount  of  the  change  in  the  portfolio’s  total  procurement  cost  could  be 
attributable  to  factors  other  than  quantity  change  such  as  production 
problems,  inefficiencies,  or  flawed  initial  cost  estimates.  For  example, 
the  F-35  program  requires  $107  billion  in  future  funding  to  cover  cost 
growth,  even  though  it  will  procure  409  fewer  aircraft  than  originally 
planned.  Two-thirds  of  all  remaining  funding — or  $463  billion — is 
required  by  the  10  programs  with  the  highest  future  funding  needs  and 
F-35  alone  requires  about  one-third  of  all  funding  to  go.  The  other  70 
programs  have  only  $218.8  billion  in  costs  remaining,  less  than  the 
amount  of  future  funding  needed  for  the  F-35  program.  Table  6  shows 
the  amount  of  future  funding  needed  as  well  as  how  much  of  that 
funding  represents  cost  growth  from  first  full  estimates. 


Table  6:  Future  Funding  and  Future  Funding  Required  Due  to  Cost  Growth  from 

First  Full  Estimates 

Fiscal  year  2014  dollars  in  billions 

Program  name 

Future  funding 
required 

Amount  of  future 
funding  due  to 
cost  growth  from 
first  full  estimates 

F-35  Joint  Strike  Fighter 

$239.6 

$107.0 

Evolved  Expendable  Launch  Vehicle 

42.9 

38.4 

KC-46  Tanker 

40.8 

0 

Virginia  Class  Submarine 

28.2 

16.2 

Joint  Light  Tactical  Vehicle 

23.0 

0 

CH-53K  Super  Stallion 

20.7 

7.4 

Littoral  Combat  Ship 

19.6 

19.6 

CVN  78  Gerald  R.  Ford  Class  Aircraft  Carrier 

17.5 

0 

P-8A  Poseidon 

16.8 

1.7 

DDG  51  Arleigh  Burke  Class  Destroyer 

14.0 

14.0 

10  programs  with  the  largest  future 
funding  requirements 

$463.2 

$204.3 

Totals  for  rest  of  portfolio 

$218.8 

$107.3 

2013  portfolio  total 

$682.0 

$311.6 

Source:  GAO  analysis  of  DOD  data. 
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Notes:  Future  funding  is  measured  as  all  fiscal  year  2014  funding  to  completion  of  procurement.  Some 
numbers  may  not  sum  due  to  rounding. 


Observations  from  Our 
Assessment  of 
Knowledge  Attained  by 
Programs  at  Key 
Acquisition  Junctures 


Our  2014  assessment  shows  that  DOD  continues  to  show  progress  in 
following  a  knowledge-based  approach  to  reduce  risk;  however,  it  has 
significant  room  for  improvement.  While  programs  that  have  recently 
passed  through  major  decision  points  have  demonstrated  best  practices — 
such  as  constraining  development  times  and  achieving  design  stability — 
key  practices  like  demonstrating  technology  maturity  or  controlling 
manufacturing  processes  are  still  not  being  fully  implemented.  As  a  result, 
many  programs  will  carry  risk  into  subsequent  phases  of  acquisition  that 
could  result  in  cost  growth  or  schedule  delays. 


Positive  acquisition  outcomes  require  the  use  of  a  knowledge-based 
approach  to  product  development  that  demonstrates  high  levels  of 
knowledge  before  significant  commitments  are  made.  In  essence, 
knowledge  supplants  risk  over  time.  In  our  past  work  examining  weapon 
acquisition  and  best  practices  for  product  development,  we  have  found 
that  leading  commercial  firms  and  successful  DOD  programs  pursue  an 
acquisition  approach  that  is  anchored  in  knowledge,  whereby  high  levels 
of  product  knowledge  are  demonstrated  at  critical  points  in  the  acquisition 
process.  This  work  led  to  multiple  recommendations  that  DOD  generally  or 
partially  agreed  with  and  has  made  progress  in  implementing.7  On  the 
basis  of  this  work,  we  have  identified  three  key  knowledge  points  during 
the  acquisition  cycle — development  start,  the  system-level  critical  design 
review,  and  production  start — at  which  programs  need  to  demonstrate 
critical  levels  of  knowledge  to  proceed.  Figure  5  aligns  the  acquisition 
milestones  described  in  DOD’s  primary  acquisition  policy  with  these 
knowledge  points.  In  this  report,  we  refer  to  DOD’s  engineering  and 


7GAO,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 
Weapon  System  Outcomes,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999);  Best 
Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better  Weapon  System 
Outcomes,  GAO-01-288  (Washington,  D.C.:  Mar.  8,  2001);  Best  Practices:  Capturing 
Design  and  Manufacturing  Knowledge  Early  Improves  Acquisition  Outcomes,  GAO-02-701 
(Washington,  D.C.:  July  15,  2002);  Defense  Acquisitions:  A  Knowledge-Based  Funding 
Approach  Could  Improve  Major  Weapon  System  Program  Outcomes,  GAO-08-619 
(Washington,  D.C.:  July  2,  2008);  Best  Practices:  High  Levels  of  Knowledge  at  Key  Points 
Differentiate  Commercial  Shipbuilding  from  Navy  Shipbuilding,  GAO-09-322  (Washington, 
D.C.:  May  13,  2009);  and  Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by 
Standardizing  the  Way  Manufacturing  Risks  Are  Managed,  GAO-10-439  (Washington,  D.C: 
Apr.  22,  2010). 
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manufacturing  development  phase  as  system  development.  Production 
start  typically  refers  to  a  program’s  entry  into  low-rate  initial  production. 


Figure  5:  DOD's  Acquisition  Cycle  and  GAO  Knowledge  Points 


Development  start  Production  start 

DOD  acquisition  milestones  (Milestone  B)  (Milestone  C) 


- System  development - 1 

Technology 

Development 

Integration 

Demonstration 

Production 

GAO  knowledge  points 


Knowledge  point  1 

Technologies  and 
resources  match 
requirements 


Knowledge  point  2 

Design  performs 
as  expected 


Knowledge  point  3 

Production  can  meet 
cost,  schedule,  and 
quality  targets 


Source:  GAO. 


The  building  of  knowledge  consists  of  information  that  should  be  gathered 
at  these  three  critical  points  over  the  course  of  a  program: 

Knowledge  point  1:  Resources  and  requirements  match.  Achieving  a 
high  level  of  technology  maturity  by  the  start  of  system  development  is  one 
of  several  important  indicators  of  whether  this  match  has  been  made.  This 
means  that  the  technologies  needed  to  meet  essential  product 
requirements  have  been  demonstrated  to  work  in  their  intended 
environment.  In  addition,  the  developer  should  complete  a  series  of 
systems  engineering  reviews  culminating  in  a  preliminary  design  of  the 
product  that  shows  the  design  is  feasible.  Constraining  the  development 
phase  of  a  program  to  5  to  6  years  is  also  recommended  because  it  aligns 
with  DOD’s  budget  planning  process  and  fosters  the  negotiation  of  trade¬ 
offs  in  requirements  and  technologies.  For  shipbuilding  programs,  critical 
technologies  should  be  matured  into  actual  system  prototypes  and 
successfully  demonstrated  in  a  realistic  environment  before  a  contract  is 
awarded  for  detail  design  of  a  new  ship. 

Knowledge  point  2:  Product  design  is  stable.  This  point  occurs  when  a 
program  determines  that  a  product’s  design  will  meet  customer 
requirements,  as  well  as  cost,  schedule,  and  reliability  targets.  A  best 
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practice  is  to  achieve  design  stability  at  the  system-level  critical  design 
review,  usually  held  midway  through  system  development.  Completion  of 
at  least  90  percent  of  engineering  drawings  at  this  point  provides  tangible 
evidence  that  the  product’s  design  is  stable,  and  a  prototype 
demonstration  shows  that  the  design  is  capable  of  meeting  performance 
requirements.  Shipbuilding  programs  should  demonstrate  design  stability 
by  completing  100  percent  of  the  basic  and  functional  drawings  as  well  as 
the  three-dimensional  product  model  by  the  start  of  construction  for  a  new 
ship.  Programs  can  also  improve  the  stability  of  their  design  by  conducting 
reliability  growth  testing  and  completing  failure  modes  and  effects 
analyses  so  fixes  can  be  incorporated  before  production  begins.  At  this 
point,  programs  should  also  begin  preparing  for  production  by  identifying 
manufacturing  risks,  key  product  characteristics,  and  critical 
manufacturing  processes. 

Knowledge  point  3:  Manufacturing  processes  are  mature.  This  point  is 
achieved  when  it  has  been  demonstrated  that  the  developer  can 
manufacture  the  product  within  cost,  schedule,  and  quality  targets.  A  best 
practice  is  to  ensure  that  all  critical  manufacturing  processes  are  in 
statistical  control — that  is,  they  are  repeatable,  sustainable,  and  capable  of 
consistently  producing  parts  within  the  product’s  quality  tolerances  and 
standards — at  the  start  of  production.  Demonstrating  critical  processes  on 
a  pilot  production  line  is  an  important  initial  step  in  this  effort.  In  addition, 
production  and  postproduction  costs  are  minimized  when  a  fully 
integrated,  capable  prototype  is  demonstrated  to  show  that  the  system  will 
work  as  intended  in  a  reliable  manner  before  committing  to  production.  We 
did  not  assess  shipbuilding  programs  for  this  knowledge  point  due  to 
differences  in  the  production  processes  used  to  build  ships. 

Knowledge  in  these  three  areas  builds  over  time.  A  knowledge  deficit  early 
in  a  program  can  cascade  through  design  and  production  leaving  decision 
makers  with  less  knowledge  to  support  decisions  about  when  and  how 
best  to  move  into  subsequent  acquisition  phases  that  commit  more 
budgetary  resources.  Demonstrating  technology  maturity  is  a  prerequisite 
for  moving  forward  into  system  development,  during  which  the  focus 
should  be  on  design  and  integration.  A  stable  and  mature  design  is 
likewise  a  prerequisite  for  moving  forward  into  production  where  the  focus 
should  be  on  efficient  manufacturing.  Additional  details  about  key 
practices  at  each  of  the  knowledge  points  can  be  found  in  appendix  IV. 

For  this  report,  we  assessed  the  knowledge  attained  by  key  junctures  in 
the  acquisition  process  for  38  current  weapon  programs,  which  are  mostly 
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in  development  or  early  production.8  Not  all  programs  included  in  our 
review  of  knowledge-based  practices  provided  information  for  every 
knowledge  point  or  had  reached  all  of  the  knowledge  points — development 
start,  design  review,  and  production  start — at  the  time  of  this  review. 

Our  analysis  of  data  from  these  38  major  defense  acquisition  programs 
allows  us  to  make  three  observations. 


Knowledge  Point  Observations 

1.  The  two  programs  that  began  system  development  in  2013 — F-22  Increment  3.2B 
and  Air  and  Missile  Defense  Radar — did  not  demonstrate  all  their  critical 
technologies  in  a  realistic  environment  and  one  of  the  two  did  not  complete  all 
relevant  systems  engineering  reviews  as  it  failed  to  hold  a  preliminary  design  review 
prior  to  system  development.  Both  programs,  however,  intend  to  constrain  their 
development  phase  to  6  years  or  less.  This  is  consistent  with  the  majority  of  the 
portfolio.  Of  the  other  programs  we  assessed,  only  4  matured  their  technologies  and 
8  completed  preliminary  design  review  in  time  for  system  development,  but  1 9 
planned  for  a  constrained  development  phase. 

2.  Of  the  three  programs  that  held  critical  design  reviews  in  2013 — KC-46  Tanker,  Joint 
Light  Tactical  Vehicle,  and  Warfighter  Information  Network-Tactical  Increment  3 — two 
demonstrated  design  stability  by  releasing  90  percent  of  planned  engineering 
drawings  and  all  three  satisfied  some  of  the  best  practices  related  to  preparation  for 
production;  but  none  tested  an  early  system-level  integrated  prototype  or 
demonstrated  all  program  critical  technologies  in  a  realistic  environment  prior  to 
design  review.  For  the  other  programs  we  assessed  that  reported  information  on  this 
review,  only  6  fully  matured  their  designs  and  3  tested  system-level  prototypes  by  the 
time  this  review  was  held.  Significant  numbers  had  conducted  activities  to  plan  for 
production. 

3.  The  two  programs  that  held  production  decisions  since  our  last  assessment — 
Paladin  Integrated  Management  and  Ground/Air  Task  Oriented  Radar — reported 
mature  technologies  and  stable  designs.  Only  one  demonstrated  performance 
through  the  testing  of  a  production-representative  prototype.  Consistent  with  our 
prior  assessments,  neither  program  has  demonstrated  that  all  their  critical 
manufacturing  processes  are  in  statistical  control,  and  only  one  program  used  a  pilot 
production  line  before  beginning  production.  Few  of  the  other  programs  in  our 
assessment  that  held  production  decisions  implemented  these  practices;  3  had 
critical  processes  in  control  and  6  had  used  a  pilot  production  line. 


Additional  detail  about  these  observations  follows. 


8Because  knowledge  points  and  best  practices  differ  for  shipbuilding  programs,  we  exclude 
the  five  shipbuilding  programs  from  some  of  our  analysis  at  each  of  the  three  knowledge 
points. 
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1.  The  two  programs  that  began  system  development  in  2013 
completed  two  systems  engineering  reviews  before  beginning 
development  and  plan  to  constrain  their  development  phase,  but 
one  program  did  not  carry  out  a  preliminary  design  review,  and 
neither  demonstrated  all  its  critical  technologies  in  a  realistic 
environment.  Both  the  F-22  Increment  3.2B  and  the  Air  and  Missile 
Defense  Radar  (AMDR)  programs  entered  system  development  with 
critical  technologies  nearing  maturity — that  is,  demonstrated  in  a 
relevant  environment — in  accordance  with  DOD  policy  and  statutory 
requirements.9  Knowledge-based  acquisition  practices,  however, 
recommend  that  programs  fully  mature  technologies  and  demonstrate 
them  in  a  realistic  or  operational  environment  prior  to  entering  system 
development.  Achieving  this  higher  level  of  technology  maturity  is  a 
better  indicator  of  whether  a  program  has  achieved  a  resource  and 
requirements  match  with  the  attainment  of  additional  knowledge  about 
the  technologies’  form,  fit,  and  function,  as  well  as  the  effect  of  the 
intended  environment  on  those  technologies.10  Neither  the  F-22 
Increment  3.2B  nor  the  AMDR  programs  satisfied  this  best  practice 
and  the  lower  maturity  level  attained  by  these  programs  is  consistent 
with  our  observations  in  past  assessments  on  programs  entering 
system  development.  Figure  6  shows  the  extent  to  which 
recommended  acquisition  practices  for  knowledge  point  1  have  been 
implemented  on  F-22  Increment  3.2B  and  AMDR  as  well  as  the  other 
36  programs  we  assessed. 


According  to  DOD  policy  at  the  time  of  our  questionnaire,  technology  should  be 
demonstrated  in  a  relevant  environment  or,  preferably,  in  an  operational  environment.  In 
addition,  a  major  defense  acquisition  program  generally  may  not  receive  approval  for 
development  start  until  the  milestone  decision  authority  certifies  that  the  technology  in  the 
program  has  been  demonstrated  in  a  relevant  environment.  10  U.S.C.  §  2366b(a)(3)(D). 

^Demonstration  in  a  relevant  environment  is  Technology  Readiness  Level  (TRL)  6. 
Demonstration  in  a  realistic  environment  is  TRL  7.  See  app.  V  for  detailed  description  of 
TRLs. 
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Figure  6:  Implementation  of  Knowledge-Based  Practices  for  Programs  in  System 
Development 
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Source:  GAO  analysis  of  DOD  data. 


Of  the  remaining  36  current  programs  we  assessed,  only  4 — Excalibur 
Precision  155mm  Projectiles  (Increment  1b),  Global  Positioning 
System  III,  LHA  6  America  Class  Amphibious  Assault  Ship,  and  Ship- 
to-Shore  Connector — reported  that  all  their  critical  technologies  were 
matured  to  best  practice  standards  when  they  began  development.11 
Another  11  programs  we  assessed  reported  having  all  critical 
technologies  nearing  maturity  prior  to  system  development.  The 
remaining  21  programs  reported  either  having  one  or  more  immature 
technologies  at  the  start  of  development  or  reported  no  information  on 
technology  maturity  at  this  critical  point  in  the  acquisition  process.  Of 


^The  MQ-9  Reaper  program  also  reported  all  critical  technologies  as  fully  mature  at 
development  start.  However,  that  technology  assessment  reflects  the  Reaper  Block  5 
configuration  and  not  the  original  program.  Our  historical  data  shows  that  the  Reaper 
program  began  development  in  2004  with  its  technologies  nearing  maturity. 
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the  1 5  programs  that  had  reported  their  critical  technologies  as  at  least 
nearing  maturity  at  development  start,  9  have  thus  far  constrained 
research  and  development  cost  growth  to  15  percent  or  less  since  first 
full  estimate.  In  contrast,  for  the  11  programs  reporting  one  or  more 
immature  critical  technologies  at  development  start,  only  2 
successfully  constrained  research  and  development  cost  growth  to  15 
percent  or  less  since  their  first  full  estimate. 

Knowledge-based  acquisition  best  practices  recommend  and  statute 
requires  that  programs  hold  systems  engineering  events,  such  as  a 
preliminary  design  review,  before  the  start  of  development  to  ensure 
that  requirements  are  defined  and  feasible  and  that  the  proposed 
design  can  meet  those  requirements  within  cost,  schedule,  and  other 
system  constraints.  The  Weapon  Systems  Acquisition  Reform  Act  of 
2009  (WSARA)  made  it  a  statutory  requirement  that  major  defense 
acquisition  programs  generally  hold  a  preliminary  design  review  before 
the  start  of  system  development.12  According  to  the  program  offices, 
the  F-22  Increment  3.2B  program  conducted  a  preliminary  design 
review  prior  to  entering  system  development,  but  the  AMDR  program 
did  not,  and  instead  sought  and  received  a  waiver  to  this  requirement. 
Currently,  AMDR  plans  to  conduct  a  preliminary  design  review  in  June 
2014.  Of  the  remaining  36  current  programs  we  assessed,  only  8  held 
a  preliminary  design  review  prior  to  development  start,  including  4 
programs  that  began  development  start  before  2009,  and  another  4 
programs  that  started  development  after  implementation  of  WSARA. 
Our  analysis  of  the  current  portfolio  shows  that  programs  that 
conducted  a  preliminary  design  review  prior  to  development  start  have, 
on  average,  experienced  a  lower  rate  of  research  and  development 
cost  growth  and  a  lower  rate  of  total  acquisition  cost  growth,  compared 
with  programs  that  did  not  have  a  preliminary  design  review  before 
development  start.  Of  the  eight  assessed  programs  that  held  this 
review  prior  to  development  start,  six  have  thus  far  constrained 
research  and  development  cost  growth  to  less  than  15  percent  since 
first  full  estimate.  Of  the  22  assessed  programs  that  did  not  carry  out  a 


12Pub.  L.  No.  111-23,  §  205(a).  A  major  defense  acquisition  program  may  not  receive 
milestone  B  approval  until  the  program  has  held  a  preliminary  design  review  and  the 
milestone  decision  authority  has  conducted  a  formal  post-preliminary  design  review 
assessment  and  certified  on  the  basis  of  such  assessment  that  the  program  demonstrates 
a  high  likelihood  of  accomplishing  its  intended  mission  unless  a  waiver  is  properly  granted 
by  the  milestone  decision  authority.  10  U.S.C.  §  2366b(a)(2),  (d)(1). 
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preliminary  design  review  prior  to  development  start,  only  8  were 
successful  in  restraining  development  cost  growth  below  15  percent. 

In  addition  to  the  preliminary  design  review,  knowledge-based 
acquisition  best  practices  recommend  the  completion  of  two  additional 
systems  engineering  reviews:  one  review  to  ensure  that  system 
requirements  have  been  properly  identified,  which  helps  ensure  mutual 
understanding  between  government  and  contractor;  and  a  system 
functional  review  to  establish  a  functional  baseline  for  the  system.  Both 
the  F-22  Increment  3.2B  and  AMDR  programs  completed  these 
reviews  prior  to  entering  system  development.  Of  the  remaining  36 
programs  we  assessed,  only  10  completed  both  reviews  prior  to 
development  start — five  that  entered  system  development  before  2009 
and  another  5  that  began  system  development  after  2009. 

Knowledge-based  acquisition  practices  also  recommend  that  a 
program  constrain  the  development  phase  to  just  5  or  6  years,  and 
both  the  F-22  Increment  3.2B  and  AMDR  programs  plan  to  do  so.13  By 
limiting  development  time  in  this  manner,  a  program  increases  the 
predictability  of  funding  needs,  as  well  as  the  likelihood  of  program 
success  by  fostering  the  negotiation  of  trade-offs  in  requirements  and 
technologies.14  For  the  remaining  31  non-ship  programs  we  assessed, 
19  had  plans  to  limit  their  development  phases  to  6  years  or  less  at  the 
time  of  development  start.  This  includes  13  out  of  23  programs  that 
began  development  prior  to  2009  and  another  6  out  of  8  programs  that 
began  system  development  during  or  after  2009. 

As  part  of  our  analysis,  we  also  assessed  18  future  programs  that 
expect  to  become  major  defense  acquisition  programs.  These 
programs  provided  information  on  the  knowledge  they  planned  to 
obtain  and  best  practices  they  intend  to  implement  before  their 
development  start  is  approved.  Seven  of  18  future  programs  reported 


13We  did  not  assess  shipbuilding  programs  against  this  recommended  practice  to  limit  the 
development  phase,  as  their  development  cycles  do  not  align  in  a  manner  consistent  with 
other  programs. 

14DOD  policy  in  place  at  the  time  of  our  review  provided  that  a  condition  for  exiting  the 
technology  development  phase  is  that  a  system  or  increment  be  developed  for  production 
within  a  short  time  frame,  defined  as  normally  less  than  5  years  for  weapon  systems. 
Department  of  Defense  Instruction  5000.02,  Operation  of  the  Defense  Acquisition  System, 
enc.  2,  para.  5.d(7)  (Dec.  8,  2008)  (hereinafter  cited  as  DODI  5000.02  (Dec.  8,  2008).  An 
interim  version  of  this  instruction  was  issued  in  November  2013. 
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that  their  technologies  are  expected  to  be  nearing  maturity;  that  is 
demonstrated  in  a  relevant  environment,  at  the  time  of  development 
start.  Ten  of  the  future  programs  plan  to  hold  a  preliminary  design 
review  before  the  start  of  development  and  only  seven  plan  to  conduct 
both  a  system  functional  and  system  requirements  review  before  the 
start  of  development.  Nine  of  the  future  programs  reported  plans  to 
limit  the  program  development  phase  to  6  years  or  less.  If  these  plans 
continue  unaltered,  programs  will  enter  system  development  with 
insufficient  knowledge  about  their  ability  to  meet  cost,  schedule,  and 
performance  requirements.  The  decision  to  proceed  without  this 
knowledge  involves  more  than  the  program  at  hand,  it  sends  signals 
across  the  portfolio  of  programs  on  what  is  acceptable.  If  programs 
that  do  not  abide  by  sound  acquisition  principles  win  funding,  then 
seeds  of  poor  outcomes  are  planted.  Top  decision  makers  must  ensure 
that  new  programs  exhibit  desirable  principles  before  they  are 
approved  and  funded;  the  proper  time  to  establish  that  is  at  program 
start. 

2.  Of  the  three  programs  that  held  critical  design  reviews  in  2013, 
two  demonstrated  design  stability  by  releasing  90  percent  of 
expected  engineering  drawings,  but  none  tested  an  early 
integrated  prototype  or  demonstrated  all  critical  technologies  in  a 
realistic  environment  prior  to  this  design  review.  Knowing  a 
product’s  design  is  stable  before  system  demonstration  reduces  the 
risk  of  costly  design  changes  as  manufacturing  begins  and 
investments  in  acquisition  become  more  significant.  Just  as  programs 
that  enter  system  development  with  immature  technologies  cost  more 
and  take  longer  to  deliver  their  operational  capabilities  to  the 
warfighter,  programs  that  hold  their  critical  design  review  before 
achieving  a  stable  design  also  experience  higher  average  costs  and 
longer  schedule  delays.  Figure  7  presents  whether  the  programs 
implemented  this,  as  well  as  other,  best  practices. 
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Figure  7:  Implementation  of  Knowledge-Based  Practices  for  Programs  at  Critical 
Design  Review 
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Of  the  three  programs  that  held  their  critical  design  review  in  2013, 
both  the  KC-46  Tanker  and  the  Joint  Light  Tactical  Vehicle  (JLTV) 
demonstrated  design  stability  by  releasing  over  90  percent  of  each 
program’s  expected  design  drawings.15  According  to  the  Warfighter 
Integration  Network-Tactical  Increment  3  (WIN-T  Inc  3)  program  office 
this  practice  was  not  applicable  as  the  system  relies  primarily  on 


15According  to  JLTV  program  officials,  a  series  of  design  understanding  reviews  were  held 
with  each  of  the  program’s  three  current  contractors  that  served  as  the  equivalent  of  a 
single  critical  design  review.  At  the  time  of  these  reviews,  all  long-lead  and  critical  designs 
had  been  completed  and  all  three  contractors  had  more  than  90  percent  of  the  design  files 
under  configuration  control. 
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software  for  its  functionality.16  Of  the  remaining  programs  we 
assessed,  30  reported  information  about  the  practices  they 
implemented  at  their  critical  design  review.  Of  these,  only  6  non-ship 
programs  released  at  least  90  percent  of  their  total  expected  design 
drawings  before  holding  this  review,  and  no  shipbuilding  programs  met 
the  shipbuilding-specific  best  practice  of  completing  100  percent  of 
their  three-dimensional  design  models  prior  to  the  start  of  fabrication. 
All  6  of  the  programs  with  a  stable  design  held  their  critical  design 
review  since  2009. 

The  use  of  early  system  prototypes  during  development  is  another 
useful  practice  for  demonstrating  that  a  system  has  a  stable  design, 
will  work  as  intended,  and  can  be  built  within  cost  and  schedule.  None 
of  the  three  programs  reported  testing  an  integrated  system  prototype 
before  their  critical  design  review  or  its  equivalent.  Instead,  the  JLTV 
program  reported  conducting  this  system  prototype  testing  7  months 
after  its  design  review,  and  the  KC-46  Tanker  program  will  not  begin 
flight  testing  a  system-level  prototype  that  integrates  the  civilian  aircraft 
and  military  sub-systems  until  January  2015,  18  months  after  its  critical 
design  review.  WIN-T  Inc  3  did  not  plan  to  test  a  system-level 
prototype  until  October  2015,  22  months  after  its  critical  design  review. 
Of  the  25  non-ship  programs  we  assessed  that  have  held  a  critical 
design  review,  21  report  they  tested  or  plan  to  test  an  early  integrated 
prototype  but  only  3  programs  conducted  this  testing  before  their 
critical  design  review.17  For  the  other  18  programs,  prototype  testing 
followed  the  review  by  an  average  of  nearly  3  years  (or  33  months). 
This  use  of  early  system  prototypes  among  programs  we  assessed  is 
much  the  same  as  past  assessments. 

Reliability  growth  testing  provides  visibility  over  how  reliability  is 
improving  and  uncovers  design  problems  so  fixes  can  be  incorporated 
before  production  begins.  All  three  programs  reported  establishing  a 
reliability  growth  curve  by  their  critical  design  review.  Of  the  remaining 


16While  the  WIN-T  Inc  3  program  held  a  critical  design  review  in  December  of  2013  that 
assessed  the  entirety  of  the  developmental  program,  it  later  had  its  requirements  and 
capabilities  reduced  through  descoping  options  recommended  at  a  configuration  steering 
board  review  and  approved  by  the  Army  acquisition  executive  in  January  2014. 

17We  did  not  assess  the  five  shipbuilding  programs  against  this  recommended  practice  as 
testing  early  system  prototypes  in  these  programs  may  not  be  practical. 
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30  programs  we  assessed  that  have  held  critical  design  review,  or  its 
equivalent,  14  had  a  reliability  growth  curve  at  that  time. 

The  programs  we  assessed  also  reported  use  of  other  knowledge- 
based  practices  to  increase  confidence  in  the  stability  of  their  product’s 
design.  Those  practices  include  the  identifying  key  product 
characteristics;  identifying  critical  manufacturing  processes; 
conducting  producibility  assessments  to  identify  manufacturing  risks; 
and  completing  failure  modes  and  effects  analysis  to  identify  potential 
failures  and  early  design  fixes.  The  KC-46  Tanker  program  carried  out 
all  four  of  these  practices,  while  the  JLTV  program  carried  out  three 
and  the  WIN-T  Inc  3  program  implemented  two.  For  the  30  other 
programs  we  assessed  with  a  critical  design  review  prior  to  2013,  a 
little  more  than  half  of  them  reported  having  used  all  four  of  these 
practices.  This  rate  of  implementation  is  similar  to  what  we  observed  in 
our  prior  year  assessment. 

3.  The  two  programs  that  held  production  decisions  since  our  last 
assessment  reported  mature  technologies  and  stable  designs, 
and  one  of  the  two  demonstrated  performance  through  the 
testing  of  a  production-representative  prototype;  neither 
demonstrated  that  all  their  manufacturing  process  capabilities 
were  in  control.  Capturing  critical  manufacturing  knowledge  before 
entering  production  helps  ensure  that  a  weapon  system  will  work  as 
intended  and  can  be  manufactured  efficiently  to  meet  cost,  schedule, 
and  quality  targets.  This  knowledge  can  be  captured  and 
demonstrated  through  the  use  of  various  proactive  methods,  including 
the  use  of  statistical  process  control  data,  pilot  production  lines, 
manufacturing  readiness  levels,  and  prototype  testing.  Figure  8  shows 
the  extent  to  which  programs  that  have  held  a  production  decision 
have  implemented  associated  knowledge-based  practices. 
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Figure  8:  Implementation  of  Knowledge-Based  Practices  for  Programs  at 
Production  Decision 
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Neither  the  Paladin  Integrated  Management  (PIM)  program,  which 
held  a  production  decision  in  2013,  nor  the  G/ATOR  program,  which 
held  a  production  decision  in  January  2014,  demonstrated  that  their 
manufacturing  process  capabilities  were  in  control  at  production  start.18 
Of  the  remaining  programs  we  assessed,  16  non-ship  programs  held 
production  decisions  prior  to  2013,  of  which  only  3  provided  data 
indicating  that  critical  manufacturing  processes  were  in  control  at 
production  start.19  Though  the  PIM  program  did  not  test  a  pilot 
production  line  prior  to  production  start,  the  G/ATOR  program  reported 
that  it  has.  Of  the  16  programs  that  held  production  decisions  prior  to 


18To  determine  whether  critical  processes  are  in  control,  we  used  program  reported  process 
capability  index  data  and  manufacturing  readiness  level  assessments  of  the  process 
capability  and  control  sub-thread. 

19The  five  shipbuilding  programs  we  assessed  were  excluded  from  this  portion  of  our 
analysis  due  to  the  differences  in  the  production  processes  used  to  build  ships. 
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2013,  only  6  have  demonstrated  manufacturing  processes  on  a  pilot 
production  line.  Another  14  programs  we  assessed  plan  to  hold  a 
production  decision  in  the  future  and  only  6  indicated  that  they  intend 
to  test  a  pilot  production  line  before  production  start. 

Production  and  postproduction  costs  are  also  minimized  when  a  fully 
integrated,  production-representative  prototype  is  demonstrated  to 
show  that  the  system  will  work  as  intended  in  a  reliable  manner.  DOD 
policy  has  also  required  that  a  system  be  demonstrated  in  its  intended 
environment  using  a  production-representative  article  before  entering 
production,  which  has  led  to  an  increase  in  the  number  of  programs 
doing  so.20  The  G/ATOR  program  has  tested  a  production- 
representative  prototype  in  its  intended  environment  in  advance  of 
their  production  decision,  but  PIM  has  not.  Of  the  16  programs  we 
assessed  that  held  a  production  decision  prior  to  2013,  only  6  tested 
production-representative  prototypes  prior  to  their  production  start.  For 
the  other  14  programs  we  assessed  that  plan  to  hold  a  production 
decision  in  the  future,  only  5  currently  have  plans  to  test  a  production- 
representative  prototype  before  reaching  this  key  juncture.  In  not 
testing  such  prototypes  these  programs  risk  repeating  the  experiences 
of  the  past  when  issues  were  discovered  late  in  testing,  triggering  the 
need  for  expensive  re-tooling  of  production  lines  and  retrofitting  of 
production  articles. 


Observations  about 
DOD’s  Implementation 
of  Key  Acquisition 
Reform  Initiatives  and 
Program  Concurrency 


During  the  past  few  years,  the  Congress  and  DOD  have  made  policy 
changes  and  begun  other  reform  initiatives  to  improve  the  way  the 
department  acquires  major  weapon  systems  and  address  the  symptoms  of 
an  acquisition  system  that  include  frequent  program  cost  overruns  and 
delays  in  operational  capability  delivery.  More  specifically,  the  enactment 
of  the  Weapon  Systems  Acquisition  Reform  Act  of  2009,  the  promulgation 
of  multiple  “Better  Buying  Power”  memorandums  by  the  Under  Secretary 
of  Defense  for  Acquisition,  Technology,  and  Logistics,  and  the  revision  of 
DOD  Instruction  5000.02  in  2008  and  interim  revision  in  2013,  represent 
efforts  to  increase  competition  in  weapon  programs,  decrease  acquisition 


20DODI  5000.02,  enc.  2,  para.  6.c(6)(d)  (Dec.  8,  2008). 
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costs,  and  ensure  that  programs  are  more  affordable.21  We  focused  our 
analysis  on  efforts  within  the  “Better  Buying  Power”  memorandums, 
particularly  affordability  constraints  and  “should-cost”  analysis.  These 
efforts  attempt  to  control  program  costs  by  introducing,  respectively, 
constraints  on  total  cost  and  a  review  process  to  eliminate  unnecessary 
costs  from  contracts.  We  also  analyzed  the  implementation  of  two  reforms 
addressed  in  legislation,  the  requirement  to  increase  competition 
throughout  the  acquisition  life  cycle  and  the  requirement  that  major 
defense  acquisition  programs  hold  annual  configuration  steering  board 
reviews  to  consider  changes  regarding  program  requirements  and 
maximize  capabilities.  GAO  has  previously  made  recommendations  or 
observations  on  the  implementation  of  a  number  of  these  reform  initiatives 
and  DOD,  which  has  generally  concurred  or  partially  concurred  with  the 
recommendations,  is  working  to  implement  them.22  In  addition,  we 
evaluated  the  amount  of  concurrency  between  developmental  testing  and 
production  for  programs  we  assessed  that  have  started  production  as  well 
as  those  that  plan  to  begin  production  in  the  coming  years.23  We  have 
consistently  emphasized  the  importance  of  completing  developmental 


21Pub.  L.  No.  111-23.  Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology, 
and  Logistics  Memorandum:  “Better  Buying  Power:  Mandate  for  Restoring  Affordability  and 
Productivity  in  Defense  Spending”  (June  28,  2010).  Office  of  the  Under  Secretary  of 
Defense,  Acquisition,  Technology,  and  Logistics  Memorandum:  “Better  Buying  Power: 
Guidance  for  Obtaining  Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Sept.  14, 
2010).  Office  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics 
Memorandum:  “Implementation  Directive  for  Better  Buying  Power  -  Obtaining  Greater 
Efficiency  and  Productivity  in  Defense  Spending”  (Nov.  3,  2010).  Office  of  the  Under 
Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics  Memorandum:  “Better  Buying 
Power  2.0:  Continuing  the  Pursuit  for  Greater  Efficiency  and  Productivity  in  Defense 
Spending”  (Nov.  13,  2012).  DODI  5000.02  (Dec.  8,  2008)  and  Interim  DODI  5000.02, 
Operation  of  the  Defense  Acquisition  System  (Nov.  25,  2013).  As  our  data  was  collected 
prior  to  the  release  of  the  latest  revision  of  DODI  5000.02,  we  did  not  use  the  new 
instruction  as  criteria  for  our  assessments. 

22GAO,  Weapons  Acquisition  Reform:  Reform  Act  Is  Helping  DOD  Acquisition  Programs 
Reduce  Risk,  but  Implementation  Challenges  Remain,  GAO-13-103  (Washington,  D.C.: 
Dec.  14,  2012);  Defense  Acquisitions:  DOD  Can  Improve  Its  Management  of  Configuration 
Steering  Boards,  GAO-11-640  (Washington,  D.C.:  July  7,  2011);  Defense  Acquisitions: 
Assessments  of  Selected  Weapon  Programs,  GAO-1 3-294SP  (Washington,  D.C.:  Mar.  28, 
2013). 

23This  analysis  reflects  30  non-ship  programs  out  of  the  38  programs  we  assessed  and 
does  not  include  three  non-ship  programs  for  which  a  production  start  date  was  not 
identified  as  well  as  five  shipbuilding  programs.  Shipbuilding  programs  are  excluded  from 
this  analysis  as  we  assess  production  knowledge  for  ships  differently. 
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testing  before  entering  production,  and  pointed  out  the  increased  risks 
associated  with  concurrent  testing  and  production.24 

To  gain  some  perspectives  on  the  implementation  of  key  aspects  of  policy 
changes  and  reform  initiatives  within  DOD’s  current  portfolio,  we  analyzed 
questionnaire  data  collected  from  38  current  major  defense  acquisition 
programs — the  same  programs  reflected  in  our  knowledge  point 
analysis — and  18  additional  programs  identified  by  DOD  as  future  major 
defense  acquisition  programs. 

Implementation  of  the  reform  initiatives  we  analyzed  varies  across 
programs.  The  department  has  largely  been  successful  in  implementing  its 
“should-cost”  initiative  with  significant  cost  savings  being  reported.  In 
addition,  many  programs  reported  holding  configuration  steering  board 
reviews.  In  contrast,  the  department  has  had  less  success  in  implementing 
affordability  constraints  and  utilizing  measures  to  ensure  competition 
before  and  after  development  start — initiatives  that  also  have  the  potential 
to  result  in  savings.  In  addition,  many  programs  are  committing  to 
production  prior  to  completing  the  developmental  testing  necessary  to 
assure  that  the  system  will  function  as  intended,  introducing  unnecessary 
risk  to  cost  and  schedule.  Our  current  analysis  allows  us  to  make  the 
following  five  observations  concerning  key  acquisition  reform  initiatives 
and  program  concurrency. 


24GAO,  Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance  Restructuring  and 
Address  Affordability  Risks,  GAO-12-437  (Washington,  D.C.:  June  14,  2012);  Missile 
Defense:  Opportunity  Exists  to  Strengthen  Acquisitions  by  Reducing  Concurrency,  GAO- 
12-486  (Washington,  D.C.:  Apr.  20,  2012);  and  Best  Practices:  Capturing  Design  and 
Manufacturing  Knowledge  Early  Improves  Acquisition  Outcomes,  GAO-02-701 
(Washington,  D.C.:  July  15,  2002). 
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Acquisition  Reform  and  Concurrency  Observations 

1 .  Only  30  of  the  56  current  and  future  programs  we  assessed  have  established  an 
affordability  constraint,  an  implementation  rate  that  has  not  improved  from  our  prior 
year  assessment,  but  all  programs  with  a  cap  that  we  assessed  reported  they  are  on 
track  to  meet  it. 

2.  About  82  percent  of  the  current  programs  we  assessed  have  conducted  a  “should- 
cost”  analysis  resulting  in  anticipated  savings  of  approximately  $24  billion;  over  half 
of  this  amount  has  or  will  be  reallocated  from  these  programs  for  other  priorities. 

3.  Many  of  the  future  programs  we  assessed  do  not  plan  to  conduct  competitive 
prototyping  before  development  start  and  many  current  programs  do  not  have 
acquisition  strategies  that  ensure  competition  through  the  end  of  production.  Fifteen 
future  and  current  programs  reported  they  will  not  take  actions  to  promote  any 
competitive  measures  before  or  after  development  start. 

4.  All  but  1  of  the  38  current  programs  we  assessed  had  conducted  a  configuration 
steering  board  review  with  29  programs  reporting  that  this  review  occurred  during  the 
past  year.  Only  10  programs  reported  that  changes  were  approved  or  recommended 
at  their  last  review. 

5.  Fifteen  of  the  18  current  programs  we  assessed  that  have  started  production  plan  to 
perform  30  percent  or  more  of  their  developmental  testing  during  production  despite 
the  increased  risk  of  design  changes  and  costly  retrofits.  Five  of  these  programs 
expect  to  place  substantial  procurement  quantities  under  contract  before 
developmental  testing  is  completed. 


Additional  details  about  each  observation  follow. 

1.  Only  30  of  the  56  current  and  future  programs  we  assessed  have 
established  an  affordability  constraint,  an  implementation  rate 
that  has  not  improved  from  our  prior  year  assessment,  but  all 
programs  with  a  cap  that  we  assessed  reported  they  are  on  track 
to  meet  it.  In  September  201 0,  the  first  of  several  memorandums  were 
issued  by  the  Under  Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  directing  programs  to  consider  affordability  a  key 
program  requirement  and  outlining  how  this  initiative  was  to  be 
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implemented  across  the  department.25  The  memorandums  emphasize 
the  need  to  treat  affordability,  defined  as  conducting  a  program  at  a 
cost  constrained  by  the  resources  that  DOD  can  allocate,  as  a  key 
requirement  at  the  start  of  development.  Affordability  constraints  are 
intended  to  force  prioritization  of  requirements,  enable  cost  trades  and 
ensure  that  unaffordable  programs  do  not  enter  the  acquisition 
process.26  More  than  half  of  the  current  and  future  programs  (30  of  56) 
we  assessed  have  established  an  affordability  constraint  as  directed 
by  the  “Better  Buying  Power”  initiatives  and  all  these  programs 
reported  that  they  are  on  track  to  meet  their  established  constraint. 
The  rate  of  implementation  of  this  reform  has  not  improved  from  the 
level  reported  in  our  prior  assessment.  For  the  38  current  programs,  1 8 
have  not  established  an  affordability  cap  and  almost  all  of  these 
programs  are  nearing  a  production  decision  or  are  already  in 
production.  In  general,  these  18  programs  are  older  and  have  smaller 
percentages  of  funding  remaining  than  those  current  programs  with  an 
affordability  cap.  Similarly,  for  the  18  future  programs,  8  reported  a  lack 
of  an  established  affordability  goal.  Two  of  them  reported  holding  a 
milestone  A  review  during  2013 — where  an  affordability  goal  would 
normally  be  established — while  a  third  program  held  its  milestone  A 
review  in  2009  before  the  affordability  requirement  was  established  by 
DOD  policy  and  the  remaining  5  responded  that  the  timeframe  for  this 
review  was  “to  be  determined”  or  “not  applicable.” 

2.  About  82  percent  of  the  current  programs  we  assessed  have 
conducted  a  “should-cost”  analysis  resulting  in  anticipated 


250ffice  of  the  Under  Secretary  of  Defense,  Acquisition,  Technology,  and  Logistics 
Memorandum:  “Better  Buying  Power:  Guidance  for  Obtaining  Greater  Efficiency  and 
Productivity  in  Defense  Spending”  (Sept.  14,  2010).  Office  of  the  Under  Secretary  of 
Defense,  Acquisition,  Technology,  and  Logistics  Memorandum:  “Implementation  Directive 
for  Better  Buying  Power  -  Obtaining  Greater  Efficiency  and  Productivity  in  Defense 
Spending”  (Nov.  3,  2010).  Office  of  the  Under  Secretary  of  Defense,  Acquisition, 
Technology,  and  Logistics  Memorandum:  “Better  Buying  Power  2.0:  Continuing  the  Pursuit 
for  Greater  Efficiency  and  Productivity  in  Defense  Spending”  (Nov.  13,  2012).  Office  of  the 
Undersecretary  of  Defense,  Acquisition,  Technology,  and  Logistics  Memorandum: 
“Implementation  Directive  on  Better  Buying  Power  2.0:  Achieving  Greater  Efficiency  and 
Productivity  in  Defense  Spending”  (Apr.  24,  2013). 

26Affo rd a b i I ity  goals  are  established  at  milestone  A,  the  entry  into  technology  development. 
After  systems  engineering  trade-offs  are  completed  during  the  technology  development 
phase,  these  affordability  goals  then  become  affordability  caps  at  milestone  B,  the  start  of 
system  development,  when  a  match  is  to  be  made  between  requirements  and  resources. 
We  refer  to  the  goals  and  caps  collectively  as  affordability  constraints. 
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savings  of  approximately  $24  billion;  over  half  of  this  amount  has 
or  will  be  pulled  from  these  programs  for  other  priorities.  DOD’s 
“Better  Buying  Power”  initiatives  also  emphasize  the  importance  of 
driving  cost  improvements  during  contract  negotiation  and  program 
execution  to  control  costs  both  in  the  short-term  and  throughout  the 
product  life  cycle.  In  accordance  with  direction  provided  in 
memorandums  issued  by  the  Under  Secretary,  each  program  must 
conduct  a  “should-cost”  analysis  which  includes  justifying  each  cost 
under  the  program’s  control  with  the  aim  of  reducing  negotiated  prices 
for  contracts  and  obtaining  other  efficiencies  in  program  execution. 
More  broadly,  “should-cost”  is  the  concept  that  program  managers  are 
to  proactively  control  expenditures  by  setting  cost  targets  below  what 
has  been  budgeted  for  their  program’s  activities  and  manage  with  the 
intent  to  achieve  these  lower  targets. 

According  to  our  analysis  of  questionnaire  responses,  32  of  38  current 
programs  we  assessed  conducted  a  “should-cost”  analysis  as  directed 
by  the  “Better  Buying  Power”  initiatives  and  31  of  the  programs 
reported  that  they  had  realized  or  expect  to  realize  cost  savings 
totaling  approximately  $24  billion  in  fiscal  year  2014  dollars  as  shown 
in  figure  9  below.27 


270ne  of  the  32  programs  that  conducted  a  “should-cost'’  analysis — the  LCS  program — 
reported  “should-cost”  savings  of  $5.8  billion  in  then-year  dollars,  which  were  not 
comparable  to  the  base  year  data  we  received  from  other  programs  and  is  not  reflected  in 
the  $24  billion  fiscal  year  2014  constant  dollar  amount. 
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Figure  9:  Realized  and  Expected  “Should-Cost”  Savings  Reported  by  31  Programs 


2  programs  where 
all  cost  savings 
have  been  realized 

19  programs  that  have 
realized  some  cost 
savings  and  expect  more 

10  programs  that  expect 
all  savings  to  be 
realized  in  the  future 


Fiscal  year  2014  dollars  (in  billions) 


Expected 

Realized 


Source:  GAO  analysis  of  questionnaire  data. 


Approximately  41  percent,  $9.9  billion,  of  all  the  “should-cost”  savings 
were  identified  as  realized,  with  the  other  59  percent,  or  $14.1  billion, 
expected  in  the  future.  When  savings  were  converted  into  fiscal  year 
2014  constant  dollars,  6  of  the  programs  reported  savings  of  more 
than  $1  billion,  the  largest  of  which  was  the  KC-46  Tanker  program 
which  reported  a  savings  total  of  $6.8  billion  with  nearly  $6.4  billion  of 
this  amount  already  realized.  For  the  6  current  programs  that  reported 
they  did  not  conduct  a  “should-cost”  analysis,  3  were  being 
restructured  during  our  assessment  after  reporting  an  increase  in  cost 
or  schedule  above  their  acquisition  program  baselines,  and  another 
had  recently  entered  system  development. 

The  1 8  programs  that  reported  some  or  all  of  their  “should-cost” 
savings  as  realized  cited  several  activities  as  being  responsible 
including 

•  improved  vendor/supply  chain  management  (10  programs), 

•  developmental  or  operational  testing  efficiencies  (12  programs),  and 

•  design  trades  conducted  to  balance  affordability  and  capability  (12 
programs). 

As  a  specific  example,  the  DDG  51  program  reported  that  nine 
different  activities  were  carried  out  to  obtain  approximately  $506  million 
in  realized  cost  savings  including  the  three  activities  above,  while  the 
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JLTV  program  cited  two  activities — design  trades  balancing 
affordability  and  capability  and  changes  in  requirements — to  achieve 
$206  million  in  realized  cost  savings  with  another  $412  million  in 
savings  expected  in  the  future.  These  savings  may  manifest 
themselves  in  the  program  by  lower  contract  prices,  lower  program 
cost  estimates,  and  possibly  lower  funding  requests. 

Achieving  efficiency  needs  to  be  done  with  caution  as  the  short-term 
savings  could  come  at  the  expense  of  long-term  needs.  For  example, 
six  of  the  programs  that  reported  implementing  testing  efficiencies  for 
savings  are  also  planning  to  conduct  concurrent  developmental  testing 
and  production  which  leaves  these  programs  at  risk  to  deficiencies 
being  discovered  during  production  that  could  require  substantial  and 
costly  modifications  to  systems  already  built.  Similarly,  four  programs 
that  started  production  before  2011  reported  that  changes  in  design  or 
requirements  will  result  in  some  savings,  but  carrying  out  this  “should- 
cost”  activity  after  production  start  could  contribute  to  design  instability. 
In  addition,  7  programs  stated  that  savings  are  being  realized  from  the 
increased  use  of  modeling  and  simulation  but  only  3  of  these  programs 
reported  that  all  the  models  being  used  for  operational  and  live  fire 
testing  were  officially  reviewed  and  certified  as  acceptable  for  their 
intended  use.  While  the  use  of  models  and  simulations  can  reduce  the 
time  and  resources  needed  to  conduct  the  required  testing,  using 
models  that  are  not  fully  accredited  to  meet  weapons  system  testing 
objectives  increases  the  likelihood  that  an  inappropriate  or  unsuitable 
simulation  is  being  used. 

Of  the  approximately  $24  billion  in  realized  and  anticipated  “should- 
cost”  savings,  a  total  of  $12.6  billion  was  reallocated  to  meet  other 
service  and  DOD  priorities.  For  example,  the  KC-46  Tanker  program 
reported  that  all  of  its  “should-cost”  savings,  totaling  $6.8  billion,  will  be 
reinvested  in  other  service  and  Office  of  the  Secretary  of  Defense 
priorities.  Approximately  $6.3  billion  was  reported  as  retained  by  16 
programs  we  assessed.  While  several  other  programs  reported  that 
their  savings,  a  total  of  $3.6  billion,  were  split  among  different  activities 
with  some  amounts  retained  and  some  reallocated  for  other  priorities. 
For  example,  the  F-35  program  reported  that  some  of  the  $884  million 
in  expected  savings  will  be  put  to  use  by  the  program  while  other 
amounts  will  be  reinvested  in  other  service  and  Office  of  the  Secretary 
of  Defense  priorities.  An  additional  $1.5  billion  in  savings  was  reported 
as  a  part  of  our  questionnaire  but  programs  stated  that  they  did  not 
know  or  had  not  yet  determined  how  these  funds  would  be 
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re-allocated.  The  concept  of  allowing  programs  to  keep  at  least  some 
of  their  savings  is  appropriate,  but  the  department  should  take  care  to 
use  this  incentive  effectively. 

Of  the  18  future  programs  that  we  assessed,  6  reported  that  they 
conducted  a  “should-cost”  analysis  prior  to  entering  system 
development  which  identified  approximately  $26  billion  in  realized  and 
expected  savings.  The  vast  majority  of  this  amount  was  reported  by 
the  Ohio  Replacement  and  Ground  Combat  Vehicle  (GCV)  programs 
with  $16  billion  and  $8.5  billion,  respectively,  in  “should-cost”  savings. 
The  Ohio  Replacement  program  reported  that  all  $16  billion  in 
procurement  and  other  savings  were  expected  in  the  future  and  would 
result  from  several  ongoing  affordability  initiatives  including  design  for 
producibility  initiatives  and  potentially  procuring  boats  in  a  multi-year 
procurement  block  buy  with  Virginia  Class  submarines.  In  contrast,  the 
GCV  program  reported  all  its  $8.5  billion  in  savings  as  realized.  The 
program  claims  savings  were  due  to  design  trades  to  balance 
affordability  and  capability  and  the  award  of  a  fixed-price  development 
contract  among  other  activities.  The  decision  to  eliminate  competition 
during  system  development  and  production  was  reported  as 
responsible  for  a  decrease  in  the  program’s  expected  cost. 

3.  Many  of  the  future  programs  we  assessed  do  not  plan  to  conduct 
competitive  prototyping  before  development  start  and  many 
current  programs  do  not  have  acquisition  strategies  that  ensure 
competition  through  the  end  of  production.  Fifteen  future  and 
current  programs  reported  they  will  not  take  actions  to  promote 
any  competitive  measures  before  or  after  development  start. 
Competition  is  a  critical  tool  for  achieving  the  best  return  on  the 
government’s  investment.  Major  defense  acquisition  programs  are 
required  to  provide  for  the  use  of  competitive  prototypes  before  a 
program  enters  system  development  and  have  acquisition  strategies 
that  ensure  competition  or  the  option  of  competition  throughout  the 
acquisition  life  cycle.28  Table  7  shows  the  activities  programs  report 
using  or  plan  to  use  to  ensure  competition  or  the  possibility  of 
competition. 


28Pub.  L.  No.  111-23,  §§  202,  203. 
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Table  7:  Use  of  Activities  to  Ensure  Competition  Reported  by  Future  and  Current  Programs 


Activity  to  ensure  competition  is  utilized3 

Of  the  18 
future 
programs 

Of  the  38 
current 
programs 

Competitive  prototyping  prior  to  development 

Competitive  prototyping  planned  or  conducted 

9 

16 

start 

No  competitive  prototyping  planned  or  conducted 

8 

22 

Measures  to  ensure  competition  after 
development  start  included  in  program  strategyb 

Measures  to  ensure  competition  are  planned  or  have  taken 
place 

8 

21 

No  measures  included 

8 

17 

Actions  taken  to  promote  competition  both  prior  to  and 
after  development  start 

5 

10 

No  actions  taken  to  promote  competition 

4 

11 

Source:  GAO  analysis  of  questionnaire  data. 


aOne  future  program — the  Amphibious  Combat  Vehicle — was  not  included  in  the  totals  above  for 
competitive  prototyping  as  it  did  not  know  whether  such  a  competition  would  be  conducted  prior  to 
development  start.  Two  future  programs — the  Amphibious  Combat  Vehicle  and  Indirect  Fire  Protection 
Capability  Increment  2 — were  not  counted  in  the  totals  for  measures  to  ensure  competition  as  they 
both  did  not  know  whether  their  technology  development  or  acquisition  strategies  would  call  for 
competition  between  development  start  and  the  completion  of  production. 

bWe  use  program  strategy  to  refer  to  both  technology  development  strategies  used  by  future  programs 
and  the  acquisition  strategies  used  by  current  programs. 


Only  25  of  56  current  and  future  programs  we  assessed  reported 
conducting  or  planning  for  any  form  of  competitive  prototyping  prior  to 
the  start  of  system  development.  Of  the  22  current  programs  with  no 
competitive  prototyping,  only  2  reported  that  the  requirement  was 
waived  pursuant  to  DOD  policy  implementing  WSARA.  The  other  20 
programs  that  reported  the  competitive  prototyping  requirement  was 
not  waived  either  began  development  before  December  2009,  when 
WSARA  was  implemented,  or  proceeded  directly  to  production.  The 
future  programs  we  assessed  reported  that  they  plan  to  conduct 
competitive  prototyping  at  a  slightly  higher  rate  than  the  current 
programs  we  assessed.  However,  8  future  programs  still  reported  that 
they  do  not  intend  to  conduct  competitive  prototyping.  Of  these,  7  plan 
to  seek  a  waiver  to  this  requirement  with  the  remaining  one  reporting 
that  the  requirement  to  seek  a  waiver  was  not  applicable. 

Measures  in  a  program’s  acquisition  strategy  to  ensure  competition  or 
the  option  of  competition  after  a  program  enters  system  development 
may  include  approaches  such  as  the  use  of  modular,  open 
architectures  to  enable  competition  for  upgrades  or  the  use  of 
build-to-print  approaches  to  enable  production  through  multiple 
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sources.29  But  as  shown  in  table  7  above,  only  29  of  56  current  and 
future  programs  reported  that  their  acquisition  strategies  currently 
include  measures  to  ensure  competition  after  the  start  of  development. 
Twenty-six  of  these  programs  stated  that  some  or  all  of  their  activities 
to  promote  competition  will  take  place  in  the  future. 

Overall,  only  10  current  programs  and  5  future  programs  have  taken  or 
plan  to  take  actions  to  promote  competition  both  before  and  after  the 
start  of  the  development.  In  contrast,  11  current  programs  and  4  future 
programs  reported  that  they  have  not  taken  any  actions,  nor  do  they 
plan  to  take  any,  to  ensure  competition  or  the  option  of  competition  for 
acquisition  at  any  point  after  program  initiation.  This  does  not  mean 
that  these  programs  will  have  no  competition  at  any  point  during  their 
acquisition;  only  that  they  do  not  intend  to  ensure  competition  through 
the  specific  activities  we  assessed. 

4.  All  but  one  of  the  38  current  programs  we  assessed  had 
conducted  a  configuration  steering  board  (CSB)  review,  29 
reported  that  this  review  occurred  during  the  past  year.  Only  10 
programs  reported  that  proposed  changes  were  approved  or 
recommended  at  the  most  recent  review.  According  to  statute  and 
DOD’s  acquisition  policy  at  the  time  of  our  review,  major  defense 
acquisition  programs  are  required  to  conduct  annual  configuration 
steering  boards  to  review  proposed  changes  to  the  program’s 
requirements  or  significant  technical  configuration  changes  that  may 
impact  cost  and  schedule  performance.30  Thirty-seven  current 
programs  we  assessed  had  conducted  such  a  review.  The  Air  and 
Missile  Defense  Radar  program  was  the  only  exception,  as  it  just 
started  development  in  October  2013  and  was  not  required  to 
complete  the  review  by  the  time  of  our  assessment.  A  majority,  29  out 
of  37  programs,  reported  that  this  review  occurred  in  the  12  months 
prior  to  the  submission  of  our  questionnaire. 


290pen  systems  architecture  is  a  design  approach  that  includes  standard  interfaces  and  the 
use  of  modular  components  within  a  product  (like  a  computer)  that  can  be  replaced  easily. 
This  allows  the  product  to  be  refreshed  with  new,  improved  components  made  by  a  variety 
of  suppliers. 

30Duncan  Hunter  National  Defense  Authorization  Act  for  Fiscal  Year  2009,  Pub.  L.  No.  110- 
417,  §  814  (2008). 
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Only  12  of  the  current  programs  we  assessed  reported  presenting  any 
changes  at  their  last  CSB  review.  Of  these,  1 0  programs  had  proposed 
requirements  changes,  configuration  changes,  descoping  options,  or 
some  combination  of  these,  which  they  reported  were  approved  or 
recommended  for  approval.  For  example,  the  DDG  1000  Zumwalt 
Class  Destroyer  program  reported  that  both  configuration  changes  and 
descoping  options  were  approved  or  recommended  for  approval  at 
their  June  2012  review,  while  the  F-35  program  reported  that 
requirements  changes  were  approved  or  recommended  at  their  March 
2013  review.  However,  some  changes  to  capabilities  have  occurred 
outside  of  the  CSB  process.  For  the  38  current  programs  we 
assessed,  14  reported  that  planned  system  capabilities  have  either 
increased  or  decreased  since  the  start  of  development.  Only  5  of  these 
programs  indicated  that  a  CSB  review  was  one  of  the  contributing 
reasons  for  the  change.  Many  of  these  14  programs  reported  that 
requirements  changes  since  development  start  have  caused  cost  and 
schedule  changes  as  well. 

5.  Fifteen  of  the  18  current  programs  we  assessed  that  have  started 
production  plan  to  perform  30  percent  or  more  of  their 
developmental  testing  during  production  despite  the  increased 
risk  of  design  changes  and  costly  retrofits.  Five  of  these 
programs  expect  to  place  substantial  procurement  quantities 
under  contract  before  developmental  testing  is  completed. 
Beginning  production  before  demonstrating  that  a  design  is  mature 
and  that  a  system  will  work  as  intended  increases  the  risk  of 
discovering  deficiencies  during  production  that  could  require 
substantial  design  changes  and  costly  modifications  to  systems 
already  built.  The  intent  of  developmental  testing  is  to  demonstrate  the 
maturity  of  a  design  and  to  discover  and  fix  design  and  performance 
problems  before  a  system  enters  production.  However,  15  out  of  the 
18  programs  we  assessed  that  have  started  production  intend  to  or 
have  already  executed  30  percent  or  more  of  their  developmental 
testing  concurrent  with  production.  Seven  of  these  15  programs  have 
completed  developmental  testing  with  4  of  them  reporting  quality 
problems  during  production. 

Five  of  the  other  8  programs  currently  conducting  concurrent  testing 
and  production,  expect  to  have  more  than  10  percent  of  their 
procurement  quantities  under  contract  before  developmental  testing 
completes.  The  F-35  program  in  particular  plans  to  have  530  aircraft, 
more  than  20  percent  of  its  total  procurement  quantity,  under  contract 
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at  a  cost  of  approximately  $57.8  billion  before  developmental  testing  is 
completed  in  201 7. 31  When  considering  the  point  at  which  budgets  to 
support  those  procurements  are  typically  approved — up  to  18  months 
in  advance — it  places  the  commitment  of  funding  that  much  further 
ahead  of  the  completion  of  developmental  testing. 

Another  twelve  programs  we  assessed  are  scheduled  to  make  a 
production  decision  in  the  coming  years  and  6  of  them  intend  to 
execute  30  percent  or  more  of  their  developmental  testing  concurrent 
with  production.  Two  out  of  these  six  programs  expect  to  have  more 
than  10  percent  of  their  total  procurement  quantity  under  contract 
before  developmental  testing  is  finished  running  the  risk  of  costly 
retrofits  to  existing  systems  or  changes  to  active  production  lines. 


Assessments  of 
Individual  Programs 


This  section  contains  assessments  of  individual  weapon  programs.  Each 
assessment  presents  data  on  the  extent  to  which  programs  are  following  a 
knowledge-based  acquisition  approach  to  product  development,  and  other 
program  information.  In  total,  we  present  information  on  56  programs.  For 
37  programs,  we  produced  two-page  assessments  discussing  the 
technology,  design,  and  manufacturing  knowledge  obtained,  as  well  as 
other  program  issues.  Each  two-page  assessment  also  contains  a 
comparison  of  total  acquisition  cost  from  the  first  full  estimate  for  the 
program  to  the  current  estimate.  The  first  full  estimate  is  generally  the  cost 
estimate  established  at  development  start;  however,  for  a  few  programs 
that  did  not  have  such  an  estimate,  we  used  the  estimate  at  production 
start  instead.  For  shipbuilding  programs,  we  used  their  planning  estimates 
if  those  estimates  were  available.  For  programs  that  began  as  non-major 
defense  acquisition  programs,  we  used  the  first  full  estimate  available. 
Thirty-four  of  these  37  two-page  assessments  are  of  major  defense 
acquisition  programs,  most  of  which  are  in  development  or  early 
production  and  3  assessments  are  of  programs  that  were  projected  to 
become  major  defense  acquisition  programs  during  or  soon  after  our 
review.  See  figure  10  for  an  illustration  of  the  layout  of  each  two-page 
assessment.  In  addition,  we  produced  one-page  assessments  on  the 


31The  cost  of  the  F-35  aircraft  under  contract  before  the  completion  of  developmental 
testing  was  calculated  using  the  average  procurement  unit  cost  for  both  the  airframe  and 
engine  as  reported  in  the  program’s  December  2012  Selected  Acquisition  Report.  This  unit 
cost  estimate  assumes  the  quantity  benefits  from  additional  purchases  of  F-35  aircraft  by 
international  partners  and  other  foreign  military  sales. 
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current  status  of  19  programs,  which  include  15  future  major  defense 
acquisition  programs,  2  major  defense  acquisition  programs  that  are  well 
into  production,  and  2  major  defense  acquisition  programs  that  were 
recently  restructured  or  curtailed. 
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Figure  10:  Illustration  of  Program  Two-Page  Assessment 


AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 


The  AIM-9X  Block  II  Is  a  Navy-led  program  to 
acquire  short-range  air-lo-air  missiles  for  the  F-35, 
the  Navy's  F-18,  and  the  Air  Force's  F-15,  F-16,  and 
F-22A  fighter  aircraft.  It  is  designed  to  detect, 
acquire.  Intercept  and  destroy  a  range  of  airborne 
threats  Block  II  includes  hardware  and  soltware 
upgrades  expected  to  Improve  the  range  from  which 
the  AIM-9X  can  engage  targets,  target 
discrimination,  and  interoperability.  It  was 
designated  a  major  defense  acquisition  program  in 
June  2011. 
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Program  Essentials 

Prim*  contractor;  Raytheon  Missile 

Systems 
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Procurament:  $4,167  0  million 
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Procurement  quantity  5,321 


Program  Performance  (Fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
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Acquisition  cycle  time  Imontns) 


As  of  Latest 

12/2011  09/2013 

$178,8  $224.5 
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The  AIM-9X  Block  II  entered  production  in  June 
2011  with  mature  critical  technologies,  a  stable 
design,  and  production  processes  Chat  had  been 
demonstrated  on  a  production  line,  but  were  not 
in  control  In  July  2013,  the  Navy  suspended 
operational  testing  for  the  AIM-9X  Block  II  due  to 
two  issues  with  missile  performance  According  to 
the  program  office,  one  issue  has  bean  resolved 
with  a  software  fix.  However,  the  root  cause  for 
the  second  issue,  related  lo  probability  of  kill,  a 
key  performance  requirement,  was  still  under 
investigation  dunng  our  review  The  program  has 
stopped  accepting  missiles  until  the  root  cause 
analysis  is  complete  and  the  program  determines 
what,  if  any,  fixes  to  those  missiles  may  be 
needed  The  program  also  expects  to  delay  the 
full-rate  production  decision  from  April  2014  until 
the  second  quarter  of  fiscal  year  2015 


lunls  nf  Major  Wripan  Programs 
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Common  Name:  A  4-9X  Block  II 


AIM-9X  Bloc  i  II  Program 

Y 

Technology  and  Design  Maturity 
The  AIM-9X  Block  II  entered  operational  testing  with 
its  critical  technologies  mature  and  its  design  stable. 
The  Block  II  began  as  a  pre-planned  product 
improvement  and  various  component  development 
and  integration  efforts  have  been  ongoing  since 
2004 .  According  to  the  Navy's  May  201 1  technology 
readiness  assessment,  the  Block  II  involves  the 
integration  of  mature  technologies,  including  a  new 
active  optical  target  detector/datelink,  an  upgraded 
electronics  unit,  and  new  operational  flight  software, 
among  others.  These  hardware  and  software 
upgrades  are  intended  to  improve  the  missile's  two- 
way  communication  capabilities,  its  tracking  and 
targeting,  its  range,  and  its  accuracy  The  program 
estimates  that  85  percent  of  Block  II  components 
are  unchanged  from  Block  I.  In  July  2013,  the  Navy 
suspended  operational  testing  due  to  missile 
performance  deficiencies  The  root  causes  of  the 
deficiencies  are  still  under  investigation. 

Production  Maturity 

The  AIM-9X  Block  II  began  production  in  June  2011 
with  manufacturing  processes  that  had  been 
demonstrated  on  a  pilot  production  line,  but  were 
not  in  control.  Prior  to  its  production  decision,  the 
program  concluded  that  its  manufacturing  readiness 
was  at  the  level  recommended  by  DOD  guidance, 
but  not  at  a  level  that  indicated  that  processes  were 
in  control.  According  to  the  program  office,  a 
production  readiness  review  and  manufacturing 
readiness  assessment  will  occur  prior  to  the  full-rate 
production  decision,  which  has  been  delayed  while 
the  program  corrects  deficiencies  discovered  in 
operational  testing,  Officials  expect  that  the 
missile's  manufacturing  processes  will  be  in  control 
prior  to  the  full  rate  production  decision  In  the 
meantime,  the  A1M-9X  prime  contractor  has  met  its 
internal  quality  goal  for  defects  per  unit,  another 
indicator  of  production  maturity,  for  the  last  several 
months  of  2013. 

Other  Program  issues 

The  Navy  suspended  operational  testing  for  the 
AIM-9X  Block  II  in  July  2013  after  the  program 
discovered  multiple  deficiencies  that  negatively 
affected  missile  performance.  Officials  report  this 
will  likely  delay  the  full-rate  production  decision  and 
initial  operational  capability  to  2015.  According  to 
program  officials,  one  of  the  two  issues  that  led  to 


the  suspension  of  testing  related  to  the  missile’s 
target  acquisition  time,  has  been  addressed  through 
a  software  change.  However,  the  root  cause  of  the 
second  issue  related  to  its  probability  of  kill— a  key 
performance  requirement— is  still  under 
investigation  Once  the  root  cause  is  identified,  the 
program  will  test  the  corrections  in  developmental 
testing  before  returning  to  operational  testing. 

According  to  program  officials,  they  have  stopped 
accepting  missiles  from  the  contractor  pending  the 
results  of  the  analysis,  but  production  continues. 
According  to  program  officials,  the  AIM-9X  Block  II 
program  removed  14  missiles  from  its  fiscal  year 
2013  procurement  plans  as  a  result  of  sequestration 
reductions.  The  AIM-9X  program  plans  to  pursue  a 
material  development  decision  to  initiate  a  Block  III 
acquisition  program  to  further  improve  missile 
performance  in  2014- 

Program  Office  Comments  — 

According  to  program  officials,  the  AIM-9X  Block  II 
program  is  exceeding  warfighters'  expectations  in 
every  way,  specifically,  in  the  areas  of  cost, 
schedule  and  performance.  The  AIM-9X  Block  II 
program  discovered  deficiencies  during  operational 
testing  and  has  implemented  manufacturing  and 
software  improvements  that  meet  or  exceed 
requirements.  The  program  remains  on  schedule  to 
meet  initial  operational  capability  threshold  and 
deliver  "game  changing  capability”  in  the  Beyond 
Visual  Range  Air-to-Air  Missile  employment. 
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©  Program  description 

©  Illustration  or  photo  of  system 

®  Schedule  timeline  identifying  key  dates  for  the  program  including  the 
start  of  development,  major  design  reviews,  production  decisions, 
and  planned  operational  capability 

©  Program  Essentials  Programmatic  information  including  the  prime 
contractor,  program  office  location,  and  funding  needed  to  complete 

©  Program  Performance  Cost  and  schedule  baseline  estimates  and 
the  latest  estimate  provided  as  of  January  2014 


©  Brief  summary  describing  the  program’s  implementation  of 
knowledge-based  acquisition  practices  and  its  current  status 

©  Attainment  of  Product  Knowledge  Depiction  of  selected 
knowledge-based  practices  and  the  program’s  progress  in 
attaining  that  knowledge 

©  Assessment  of  program’s  technology,  design,  and  production 
maturity,  as  well  as  other  program  issues 

O  Program  Office  Comments  General  comments  provided  by  the 
cognizant  program  office 


Source:  GAO  analysis. 
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For  our  two-page  assessments,  we  depict  the  extent  of  knowledge  gained 
in  a  program  by  the  time  of  our  review  with  a  scorecard  and  narrative 
summary  at  the  bottom  of  the  first  page  of  each  assessment.  As  illustrated 
in  figure  10  above,  the  scorecard  displays  eight  key  knowledge-based 
acquisition  practices  that  should  be  implemented  by  certain  points  in  the 
acquisition  process.  The  more  knowledge  the  program  has  attained  by 
each  of  these  key  points,  the  more  likely  the  weapon  system  will  be 
delivered  within  its  estimated  cost  and  schedule.  A  knowledge  deficit 
means  the  program  is  proceeding  without  sufficient  knowledge  about  its 
technologies,  design,  or  manufacturing  processes,  and  faces  unresolved 
risks  that  could  lead  to  cost  increases  and  schedule  delays. 

For  each  program,  we  identify  a  knowledge-based  practice  that  has  been 
implemented  with  a  closed  circle.  We  identify  a  knowledge-based  practice 
that  has  not  yet  been  implemented  with  an  open  circle.  If  the  program  did 
not  provide  us  with  enough  information  to  make  a  determination,  we  show 
this  with  a  dashed  line.  A  knowledge-based  practice  that  is  not  applicable 
to  the  program  is  grayed  out.  A  knowledge-based  practice  may  not  be 
applicable  to  a  particular  program  if  either  the  point  in  the  acquisition  cycle 
when  the  practice  should  be  implemented  has  not  yet  been  reached,  or  if 
the  particular  practice  is  not  relevant  to  the  program.  For  programs  that 
have  not  yet  entered  system  development,  we  show  a  projection  of 
knowledge  attained  for  the  first  three  practices.  For  programs  that  have 
entered  system  development  but  not  yet  held  a  critical  design  review,  we 
assess  actual  knowledge  attained  for  these  three  practices.  For  programs 
that  have  held  a  critical  design  review  but  not  yet  entered  production,  we 
assess  knowledge  attained  for  the  first  five  practices.  For  programs  that 
have  entered  production,  we  assess  knowledge  attained  for  all  eight 
practices. 

We  make  adjustments  to  both  the  key  points  in  the  acquisition  cycle  and 
the  applicable  knowledge-based  practices  for  shipbuilding  programs.  For 
shipbuilding  programs  that  have  not  yet  awarded  a  detailed  design 
contract,  we  show  a  projection  of  knowledge  attained  for  the  first  three 
practices.  For  shipbuilding  programs  that  have  awarded  this  contract  but 
not  yet  started  construction,  we  would  assess  actual  knowledge  attained 
for  these  three  practices.  For  shipbuilding  programs  that  have  started 
construction,  we  assess  the  knowledge  attained  for  the  first  four  practices. 
We  do  not  assess  the  remaining  four  practices  for  shipbuilding  programs. 
See  figure  11  for  examples  of  the  knowledge  scorecards  we  use  to  assess 
these  different  types  of  programs. 
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Figure  11:  Examples  of  Knowledge  Scorecards 

Program  in  production  Shipbuilding  program 


Attainment  of  Product  Knowledge 


As  of  January  2014 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

Q  Knowledge  attained  ■•••  Information  not  available 

O  Knowledge  not  attained  Not  applicable 


Attainment  of  Product  Knowledge 


As  of  January  2014 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  Information  not  available 

O  Knowledge  not  attained  Not  applicable 


Source:  GAO. 


Statement  on  Small 
Business  Participation 


Pursuant  to  a  mandate  in  the  report  accompanying  the  National  Defense 
Authorization  Act  for  Fiscal  Year  2013,  we  reviewed  whether  individual 
subcontracting  reports  from  a  program’s  prime  contractor  or  contractors 
were  accepted  on  the  Electronic  Subcontracting  Reporting  System 
(eSRS).32  We  reviewed  this  information  for  36  of  the  major  defense 
acquisition  programs  in  our  assessment  using  the  contract  information 
reported  in  available  Selected  Acquisition  Reports.  The  contract  numbers 
for  each  program’s  or  element’s  prime  contracts  were  entered  into  the 
eSRS  database  to  determine  whether  the  individual  subcontracting  reports 
from  the  prime  contractors  had  been  accepted  by  the  government.  The 
government  uses  individual  subcontracting  reports  on  eSRS  as  one 


32H.R.  Rep.  No.  112-479,  at  284  (2012),  accompanying  Pub.  L.  No. 112-239 
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method  of  monitoring  small  business  participation,  as  the  report  includes 
goals  for  small  business  subcontracting.  Not  all  prime  contracts  for  major 
defense  acquisition  programs  are  required  to  submit  individual 
subcontracting  reports.  For  example,  some  contractors  report  small 
business  participation  at  a  corporate  level  as  opposed  to  a  program  level 
and  this  data  is  not  captured  in  the  individual  subcontracting  reports. 
Information  gathered  for  this  analysis  is  presented  in  appendix  VI. 
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Common  Name:  AIM-9X  Block  II 


AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

The  AIM-9X  Block  II  is  a  Navy-led  program  to 
acquire  short-range  air-to-air  missiles  for  the  F-35, 
the  Navy's  F-1 8,  and  the  Air  Force's  F-1 5,  F-1 6,  and 
F-22A  fighter  aircraft.  It  is  designed  to  detect, 
acquire,  intercept  and  destroy  a  range  of  airborne 
threats.  Block  II  includes  hardware  and  software 
upgrades  expected  to  improve  the  range  from  which 
the  AIM-9X  can  engage  targets,  target 
discrimination,  and  interoperability.  It  was 
designated  a  major  defense  acquisition  program  in 
June  2011. 


Source:  U.S.  Navy. 
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Re-start  Full-rate  Initial 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Raytheon  Missile 

As  of 

Latest 

Percent 

Systems 

12/2011 

09/2013 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$178.8 

$224.5 

25.6 

Funding  needed  to  complete: 

Procurement  cost 

$4,023.0 

$3,568.0 

-11.3 

R&D:  $113.2  million 

Total  program  cost 

$4,201.8 

$3,792.5 

-9.7% 

Procurement:  $4,167.0  million 

Program  unit  cost 

$0,700 

$0,632 

-9.7% 

Total  funding:  $4,280.2  million 

Total  quantities 

6,000 

6,000 

0.0 

Procurement  quantity:  5,321 

Acquisition  cycle  time  (months) 

39 

TBD 

TBD 

The  AIM-9X  Block  II  entered  production  in  June 
2011  with  mature  critical  technologies,  a  stable 
design,  and  production  processes  that  had  been 
demonstrated  on  a  production  line,  but  were  not 
in  control.  In  July  2013,  the  Navy  suspended 
operational  testing  for  the  AIM-9X  Block  II  due  to 
two  issues  with  missile  performance.  According  to 
the  program  office,  one  issue  has  been  resolved 
with  a  software  fix.  However,  the  root  cause  for 
the  second  issue,  related  to  probability  of  kill,  a 
key  performance  requirement,  was  still  under 
investigation  during  our  review.  The  program  has 
stopped  accepting  missiles  until  the  root  cause 
analysis  is  complete  and  the  program  determines 
what,  if  any,  fixes  to  those  missiles  may  be 
needed.  The  program  also  expects  to  delay  the 
full-rate  production  decision  from  April  2014  until 
the  second  quarter  of  fiscal  year  201 5. 
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Common  Name:  AIM-9X  Block  II 


AIM-9X  Block  II  Program 

Technology  and  Design  Maturity 

The  AIM-9X  Block  II  entered  operational  testing  with 
its  critical  technologies  mature  and  its  design  stable. 
The  Block  II  began  as  a  pre-planned  product 
improvement  and  various  component  development 
and  integration  efforts  have  been  ongoing  since 
2004.  According  to  the  Navy's  May  2011  technology 
readiness  assessment,  the  Block  II  involves  the 
integration  of  mature  technologies,  including  a  new 
active  optical  target  detector/datalink,  an  upgraded 
electronics  unit,  and  new  operational  flight  software, 
among  others.  These  hardware  and  software 
upgrades  are  intended  to  improve  the  missile's  two- 
way  communication  capabilities,  its  tracking  and 
targeting,  its  range,  and  its  accuracy.  The  program 
estimates  that  85  percent  of  Block  II  components 
are  unchanged  from  Block  I.  In  July  2013,  the  Navy 
suspended  operational  testing  due  to  missile 
performance  deficiencies.  The  root  causes  of  the 
deficiencies  are  still  under  investigation. 

Production  Maturity 

The  AIM-9X  Block  II  began  production  in  June  2011 
with  manufacturing  processes  that  had  been 
demonstrated  on  a  pilot  production  line,  but  were 
not  in  control.  Prior  to  its  production  decision,  the 
program  concluded  that  its  manufacturing  readiness 
was  at  the  level  recommended  by  DOD  guidance, 
but  not  at  a  level  that  indicated  that  processes  were 
in  control.  According  to  the  program  office,  a 
production  readiness  review  and  manufacturing 
readiness  assessment  will  occur  prior  to  the  full-rate 
production  decision,  which  has  been  delayed  while 
the  program  corrects  deficiencies  discovered  in 
operational  testing.  Officials  expect  that  the 
missile's  manufacturing  processes  will  be  in  control 
prior  to  the  full  rate  production  decision.  In  the 
meantime,  the  AIM-9X  prime  contractor  has  met  its 
internal  quality  goal  for  defects  per  unit,  another 
indicator  of  production  maturity,  for  the  last  several 
months  of  2013. 

Other  Program  Issues 

The  Navy  suspended  operational  testing  for  the 
AIM-9X  Block  II  in  July  2013  after  the  program 
discovered  multiple  deficiencies  that  negatively 
affected  missile  performance.  Officials  report  this 
will  likely  delay  the  full-rate  production  decision  and 
initial  operational  capability  to  2015.  According  to 
program  officials,  one  of  the  two  issues  that  led  to 


the  suspension  of  testing  related  to  the  missile's 
target  acquisition  time,  has  been  addressed  through 
a  software  change.  However,  the  root  cause  of  the 
second  issue  related  to  its  probability  of  kill — a  key 
performance  requirement — is  still  under 
investigation.  Once  the  root  cause  is  identified,  the 
program  will  test  the  corrections  in  developmental 
testing  before  returning  to  operational  testing. 
According  to  program  officials,  they  have  stopped 
accepting  missiles  from  the  contractor  pending  the 
results  of  the  analysis,  but  production  continues. 
According  to  program  officials,  the  AIM-9X  Block  II 
program  removed  14  missiles  from  its  fiscal  year 
201 3  procurement  plans  as  a  result  of  sequestration 
reductions.  The  AIM-9X  program  plans  to  pursue  a 
material  development  decision  to  initiate  a  Block  III 
acquisition  program  to  further  improve  missile 
performance  in  2014. 

Program  Office  Comments 

According  to  program  officials,  the  AIM-9X  Block  II 
program  is  exceeding  warfighters'  expectations  in 
every  way,  specifically,  in  the  areas  of  cost, 
schedule  and  performance.  The  AIM-9X  Block  II 
program  discovered  deficiencies  during  operational 
testing  and  has  implemented  manufacturing  and 
software  improvements  that  meet  or  exceed 
requirements.  The  program  remains  on  schedule  to 
meet  initial  operational  capability  threshold  and 
deliver  "game  changing  capability"  in  the  Beyond 
Visual  Range  Air-to-Air  Missile  employment. 
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Common  Name:  AMDR 


Air  and  Missile  Defense  Radar  (AMDR) 

The  Navy's  Air  and  Missile  Defense  Radar  (AMDR) 
will  be  a  next-generation  radar  system  designed  to 
provide  ballistic  missile  defense,  air  defense,  and 
surface  warfare  capabilities.  AMDR  will  consist  of 
an  S-band  radar  for  ballistic  missile  defense  and  air 
defense,  X-band  radar  for  horizon  search,  and  a 
radar  suite  controller  that  controls  and  integrates  the 
two  radars.  AMDR  will  initially  support  DDG  51 
Flight  III.  The  Navy  expects  AMDR  to  provide  the 
foundation  for  a  scalable  radar  architecture  that  can 
be  used  to  defeat  advanced  threats. 


Source:  U.S.  Navy. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Raytheon 

™ 

Latest 

Percent 

Program  office:  Washington,  DC 

As  of 

10/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

NA 

$1,895.3 

NA 

R&D:  $881.1  million 

Procurement  cost 

NA 

$3,919.8 

NA 

Procurement:  $3,919.8  million 

Total  program  cost 

NA 

$5,844.4 

NA 

Total  funding:  $4,800.9  million 

Program  unit  cost 

NA 

$265,655 

NA 

Procurement  quantity:  22 

Total  quantities 

NA 

22 

NA 

Acquisition  cycle  time  (months) 

NA 

158 

NA 

AMDR  entered  system  development  in  October  of 
2013  with  all  four  of  its  critical  technologies 
approaching  full  maturity.  This  was  6  months  later 
than  planned,  leading  to  a  delay  in  many  of  the 
program's  future  events.  These  delays  might 
impact  the  Navy's  plan  to  design  the  DDG  51 
Flight  III,  intended  to  carry  the  radar.  The  Navy 
and  shipbuilders  have  determined  that  a  14-foot 
active  radar  is  the  largest  that  can  be 
accommodated  by  the  existing  DDG  51  though 
AMDR  is  also  being  developed  as  a  scalable 
design.  A  new  X-Band  portion  will  be  developed 
under  a  separate  program  at  a  later  date,  with 
AMDR  initially  using  a  SPQ-9B  radar  in  its  place. 


Attainment  of  Product  Knowledge 


As  of  January  2014 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  AMDR 


AMDR  Program 

Technology  and  Design  Maturity 

All  four  of  the  AMDR's  critical  technologies  are 
approaching  full  maturity  and  were  demonstrated 
using  a  1000-element  radar  array.  The  array  is  a 
smaller  version  of  the  planned  AMDR  arrays. 
According  to  the  program,  two  technologies 
previously  identified  as  the  most  challenging — 
digital-beam-forming  and  transmit-receive  modules, 
have  been  demonstrated  in  a  relevant  environment. 
Program  officials  stated  digital-beam-forming  is 
necessary  for  AMDR's  simultaneous  air  and  ballistic 
missile  defense  mission.  The  AMDR's  transmit- 
receive  modules — the  individual  radiating  elements 
of  the  radar — use  gallium  nitride  technology  instead 
of  the  legacy  gallium  arsenide  technology  for 
potential  efficiency  gains.  The  other  two  critical 
technologies  are  related  to  software  and  digital 
receivers  and  exciters.  Officials  stated  that  software 
development  will  require  a  significant  effort.  A  series 
of  software  builds  are  expected  to  deliver 
approximately  1  million  lines  of  code  and  are 
designed  to  apply  open  system  approaches  to 
commercial,  off-the-shelf  hardware.  Integrating  the 
X-band  radar  will  require  further  software 
development. 

According  to  officials,  much  of  AMDR's  hardware 
has  been  demonstrated  ahead  of  the  preliminary 
design  review,  scheduled  for  June  2014,  though  the 
number  of  design  drawings  is  still  pending.  The 
Navy  obtained  a  waiver  to  hold  this  review  after  the 
start  of  system  development  citing  cost  concerns 
and  a  belief  that  the  prototype  array  satisfactorily 
demonstrates  AMDR's  critical  technologies. 

Other  Program  Issues 

AMDR  entered  system  development  in  October  of 
2013 — 6  months  later  than  planned,  with 
corresponding  delays  to  future  program  milestones. 
These  delays  might  have  an  impact  on  the  ability  of 
the  AMDR  program  to  maintain  its  planned 
schedule  for  delivery  in  2019.  Additionally,  the 
delays  might  also  hinder  timely  delivery  of 
necessary  information  related  to  AMDR's 
parameters,  such  as  power,  cooling,  and  space 
requirements  needed  for  ongoing  and  planned 
design  studies  related  to  Flight  III  development. 
However,  program  officials  do  not  anticipate  a  delay 
for  AMDR's  planned  2019  availability  for  integration 
into  the  first  planned  DDG  51  Flight  III.  The  X-band 


portion  of  AMDR  will  be  comprised  of  an  upgraded 
version  of  an  existing  rotating  radar  (SPQ-9B), 
instead  of  the  new  design  initially  planned.  The  new 
radar  will  instead  be  developed  as  a  separate 
program  at  a  later  date  and  integrated  with  the 
thirteenth  AMDR  unit.  According  to  the  Navy,  the 
upgraded  SPQ-9B  radar  fits  better  within  the  Flight 
Ill's  sea  frame  and  expected  power  and  cooling 
availability.  Program  officials  state  that  the  SPQ-9B 
radar  will  have  capabilities  equal  to  the  new  design 
for  current  anti-air  warfare  threats,  it  will  not  perform 
as  well  against  future  threats. 

The  Navy  plans  to  install  a  14-foot  variant  of  AMDR 
on  Flight  III  DDG  51s  starting  in  2019.  According  to 
draft  AMDR  documents,  a  14-foot  radar  is  needed 
to  meet  threshold  requirements,  but  an  over  20-foot 
radar  is  required  to  fully  meet  the  Navy's  desired 
integrated  air  and  missile  defense  needs.  However, 
the  shipyards  and  the  Navy  have  determined  that  a 
14-foot  active  radar  is  the  largest  that  can  be 
accommodated  within  the  existing  DDG  51 
deckhouse.  Navy  officials  stated  that  AMDR  is 
being  developed  as  a  scalable  design  but  a  new 
ship  would  be  required  to  host  a  larger  version  of 
AMDR. 

Program  Office  Comments 

The  program  office  concurred  with  this  assessment 
and  provided  technical  comments  which  were 
incorporated  where  appropriate. 
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CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 


The  Marine  Corps'  CH-53K  heavy-lift  helicopter  is 
intended  to  transport  armored  vehicles,  equipment, 
and  personnel  to  support  operations  deep  inland 
from  a  sea-based  center  of  operations.  The  CH-53K 
is  expected  to  replace  the  legacy  CH-53E  helicopter 
and  provide  increased  range  and  payload, 
survivability  and  force  protection,  reliability  and 
maintainability,  and  coordination  with  other  assets, 
while  reducing  total  ownership  cost. 


Source:  2013  Sikorsky  Aircraft  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Sikorsky  Aircraft 

As  of 

Latest 

Percent 

Corporation 

12/2005 

08/2013 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$4,605.4 

$6,395.4 

38.9 

Funding  needed  to  complete: 

Procurement  cost 

$12,808.2 

$18,366.2 

43.4 

R&D:  $2,258.5  million 

Total  program  cost 

$17,413.6 

$24,805.5 

42.4 

Procurement:  $18,366.2  million 

Program  unit  cost 

$111,625 

$124,028 

11.1 

Total  funding:  $20,668.6  million 

Total  quantities 

156 

200 

28.2 

Procurement  quantity:  194 

Acquisition  cycle  time  (months) 

119 

157 

31.9 

The  CH-53K  program  continues  to  move  forward 
toward  production,  but  has  not  yet  fully  matured 
its  critical  technologies  or  demonstrated  that  its 
design  can  perform  as  expected.  As  a  result,  the 
risk  of  design  changes  remains.  According  to 
program  officials,  the  ground  test  vehicle  has 
been  delivered.  However,  a  number  of  problems 
with  producibility  and  qualification  of  parts  have 
resulted  in  a  delay  in  production  of  approximately 
6  months  and  delays  to  qualification  and  flight 
testing. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


Q  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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CH-53K  Program 

Technology  Maturity 

The  CH-53K  program  began  system  development 
in  2005  with  immature  critical  technologies.  Nearly  9 
years  later  the  program's  two  critical  technologies — 
the  main  rotor  blade  and  main  gearbox — are 
approaching  maturity.  The  program  expects  these 
technologies  to  be  demonstrated  in  a  realistic 
environment  by  its  planned  February  2016 
production  decision,  a  delay  in  6  months  over  last 
year's  schedule.  Program  officials  reported  that  they 
conducted  a  three-blade  whirl  test  that  produced 
results  that  exceeded  required  outcomes.  Flight 
testing  is  expected  to  begin  in  late  2014. 

Design  Maturity 

The  CH-53K  design  appears  stable,  but  it  has  not 
been  demonstrated  using  a  system-level  prototype 
to  show  that  it  will  perform  as  expected.  The 
program  has  reported  that  they  are  relying  on 
delivery  of  test  vehicles  to  demonstrate  their  design. 
However,  delivery  of  three  of  four  of  those  test 
vehicles  is  projected  to  be  delayed.  Defense 
Contract  Management  Agency  (DCMA)  officials 
reported  that  the  delays  were  caused  by  a  number 
of  factors  such  as  late  qualification  testing, 
qualification  test  failures,  re-testing,  and  the 
qualification  of  software.  The  delays  have  affected 
aircraft  test  schedules  as  well. 

Production  Maturity 

Delivery  of  the  ground  test  vehicle,  which  began 
production  in  July  2011 ,  took  place  in  October  2012. 
However,  light  off  of  the  ground  test  vehicle — which 
will  include  testing  of  the  main  gear  box  and  the 
rotor  blades — was  delayed  until  December  2013. 
Late  component  deliveries  and  parts  shortages 
have  delayed  additional  test  aircraft  deliveries.  Test 
vehicle  production  is  now  approximately  10  months 
behind  schedule,  which  has  delayed  the  program's 
first  test  flight  by  approximately  4  months.  These 
delays  also  resulted  in  a  6-month  delay  for  the  CH- 
53K's  production  decision,  which  has  been  moved 
to  February  2016. 

Other  Program  Issues 

The  CH-53K  development  contract  was  modified  in 
April  2013  to  reflect  a  new  acquisition  program 
baseline  that  reflects  long  standing  increases  in  the 
program's  cost  and  schedule.  The  program  office 
reported  that  part  of  the  increase  to  the 


development  contract  was  to  include  four  system 
demonstration  test  articles.  Program  officials 
reported  that  in  July  2011,  the  contract's  estimated 
cost  increased  by  $724  million  to  $3.4  billion. 
However,  recent  Defense  Contract  Management 
Agency  independent  cost  estimates  place  the  cost 
closer  to  $3.7  billion,  a  total  cost  increase  of 
approximately  $1  billion.  The  program's  schedule 
contains  concurrency  between  production  and 
testing,  which  represents  risk  to  the  program  and 
may  require  costly  retrofits  to  aircraft  after  they  have 
already  been  produced. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

The  Navy's  DDG  1000  destroyer  is  a  multimission 
surface  ship  designed  to  provide  advanced 
capability  for  littoral  operations  and  land-attack  in 
support  of  forces  ashore.  The  ship  will  feature  a  low 
radar  profile  and  an  advanced  gun  system.  The  lead 
ship  was  launched  in  October  2013  and  the  ship 
(comprised  of  the  hull  and  mechanical  and  electrical 
systems)  is  currently  scheduled  for  delivery  in 
September  2014,  with  activation  of  the  mission 
systems  starting  in  2015.  Fabrication  is  underway 
on  the  remaining  two  ships  in  the  class. 


Source:  ©2013  General  Dynamics  Bath  Iron  Works. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  BAE  Systems,  Bath 

As  of 

Latest 

Percent 

Iron  Works,  Huntington  Ingalls 

1/1998 

08/2013 

change 

Industries,  Raytheon 

Research  and  development  cost 

$2,395.9 

$10,593.2 

342.2 

Program  office:  Washington,  DC 

Procurement  cost 

$34,205.7 

$11,406.4 

-66.7 

Funding  needed  to  complete: 

Total  program  cost 

$36,601.6 

$21,999.6 

-39.9 

R&D:  $569.4  million 

Program  unit  cost 

$1,143,799 

$7,333,213 

541.1 

Procurement:  $768.8  million 

Total  quantities 

32 

3 

-90.6 

Total  funding:  $1,338.2  million 

Acquisition  cycle  time  (months) 

128 

222 

73.4 

Procurement  quantity:  0 

The  DDG  1000  program  has  made  progress  in 
developing  its  critical  technologies  and  the  ship 
design  is  largely  mature.  While  few  design 
changes  resulted  from  lead  ship  construction,  the 
Navy  decided  to  build  the  third  ship’s  deckhouse 
and  hangar  with  steel  rather  than  composite 
materials  due  to  manufacturing  and  integration 
challenges  on  the  first  two  ships  and  as  a  long¬ 
term  cost  saving  measure.  The  program  is  also 
incorporating  technical  configuration  changes  to 
maintain  capability  and  minimize  cost.  Delivery  of 
the  lead  ship  (hull,  mechanical  and  electrical 
systems)  may  slip  past  the  currently  scheduled 
date  of  September  2014  due  to  difficulties  in 
completing  the  ship's  electrical  systems  which  is 
impacting  test  and  activation  events. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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DDG  1000  Program 

Technology  Maturity 

The  DDG-1000  program  has  made  progress  in 
developing  its  critical  technologies.  However,  only  3 
of  11  are  mature  and  the  remaining  8  will  not  be 
demonstrated  in  a  realistic  environment  until  after 
installation  on  the  lead  ship.  Guided  flight  tests  of 
the  gun  system's  long-range  land-attack  projectile 
were  successfully  completed  in  October  2013.  After 
significant  cost  growth  and  development 
challenges,  all  six  software  releases  for  the  total 
ship  computing  environment  have  been  completed 
and  certified  to  support  lead  ship  activation  and 
delivery  and  a  software  spiral  update  is  under 
contract  timed  to  support  initial  operating  capability. 
Following  a  critical  Nunn-McCurdy  unit-cost  breach 
in  2010,  DOD  restructured  the  program  and 
removed  the  volume  search  radar  from  the  ship's 
design.  A  modified  multifunction  radar  is  expected 
to  begin  land  based  testing  in  2014,  followed  by  at- 
sea  testing  in  2015. 

Design  and  Production  Maturity 

The  DDG  1000  design  is  largely  mature.  According 
to  the  program  manager  and  ship  builder,  as  of 
October  2013,  production  and  test  efforts  for  the  first 
two  ships  were  88  and  70  percent  complete, 
respectively.  While  few  design  changes  have 
resulted  from  lead  ship  construction,  shipbuilders 
have  experienced  challenges  with  the  manufacture 
and  integration  of  the  composite  deckhouse  for  the 
first  and  second  ships,  resulting  in  rework  and 
schedule  delays  particularly  on  the  first  ship.  The 
Navy  has  emphasized  a  joint  inspection  process 
whereby  the  prime  contractors  and  the  Navy 
validate  product  quality  and  completeness  prior  to 
integration  with  the  hull.  After  assessing  alternatives 
and  conducting  a  competition,  the  Navy  decided  to 
build  the  third  ship's  deckhouse  and  hangar  with 
steel  as  a  cost  saving  measure  as  the  program 
manager  noted  that  the  Navy  has  better  cost 
insights  into  the  long-term  maintenance  of  steel 
compared  to  composite  materials. 

Other  Program  Issues 

Delivery  of  the  lead  ship  (hull,  mechanical  and 
electrical  systems)  may  slip  past  the  currently 
scheduled  date  of  September  2014  and  the  Navy  is 
in  the  process  of  assessing  the  delivery  date.  If 
delivery  slips  past  October  2014,  the  program  will 
breach  its  acquisition  program  baseline  schedule 


requirements.  According  to  the  Navy,  the  delay  is 
the  result  of  difficulties  in  completing  the  ship's 
electrical  systems,  which  is  impacting  test  and 
activation  events. 

The  program  has  awarded  all  major  contracts  for 
the  three  ships  in  the  DDG  1000  class  among  four 
prime  contractors.  As  the  integrator,  the  Navy  is 
responsible  for  ensuring  on-time  delivery  of 
products  and  bears  the  costs  of  schedule  delays 
that  affect  another  contractor.  Bath  Iron  Works  is 
now  producing  the  hull  for  all  three  ships.  In  August 
2013,  the  Navy  awarded  a  fixed  price  contract  to 
Bath  Iron  Works  for  a  steel  deckhouse,  hangar,  and 
aft  peripheral  vertical  launching  system  for  the  third 
ship.  According  to  the  program  manager,  the 
shipbuilder  negotiated  a  5-  and  10-month  delivery 
delay  for  the  second  and  third  ship,  respectively. 

Program  officials  also  reported  that  the  program 
absorbed  sequestration  reductions  of  $70  million  in 
fiscal  year  2013  by  delaying  testing  and  the  award 
of  contracts  for  mission-related  equipment.  The 
program  is  also  incorporating  technical 
configuration  changes  resulting  from  the 
configuration  steering  board  and  Nunn-McCurdy 
annual  cost  review  aimed  at  maintaining  capability 
and  minimizing  costs.  For  example,  the  program 
manager  told  us  the  program  assessed  alternatives 
to  the  close  in  gun  system  and  chose  a  legacy 
system  that  met  requirements  with  about  half  the 
weight  and  cost. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment  the 
Navy  noted  that  the  DDG  1000  lead  ship  was 
successfully  launched  in  October  201 3,  a 
christening  is  scheduled  for  April  2014,  and 
activation  activities  continue.  For  the  second  ship, 
keel  laying  and  hangar  erection  have  occurred,  hull 
integration  is  underway,  and  deckhouse  delivery  is 
scheduled  for  spring  2014.  The  Navy  noted  that 
significant  integration  efforts  between  four  prime 
contractors  continue  with  a  focus  on  cost  reduction 
and  schedule  performance.  The  Navy  also  provided 
technical  comments,  which  were  incorporated 
where  deemed  appropriate. 
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Enhanced  Polar  System  (EPS) 

The  Air  Force's  Enhanced  Polar  System  (EPS)  is  to 
provide  next-generation  protected  extremely  high 
frequency  (EHF)  satellite  communications  in  the 
polar  region.  It  will  replace  the  current  Interim  Polar 
System  and  serve  as  a  polar  adjunct  to  the 
Advanced  EHF  (AEHF)  system.  EPS  is  to  consist  of 
two  EHF  payloads  hosted  on  classified  satellites,  a 
gateway  to  connect  modified  Navy  Multiband 
Terminals  to  other  communication  systems,  and  a 
control  and  planning  segment  (CAPS). 


Source:  LinQuest  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

Latest 

Percent 

Information  Systems 

As  of 

07/2013 

change 

Program  office:  Los  Angeles  AFB,  CA 

Research  and  development  cost 

NA 

$1,505.3 

NA 

Funding  needed  to  complete: 

Procurement  cost 

NA 

$0.0 

NA 

R&D:  $398.1  million 

Total  program  cost 

NA 

$1,505.3 

NA 

Procurement:  $0.0  million 

Program  unit  cost 

NA 

$752,625 

NA 

Total  funding:  $398.1  million 

Total  quantities 

NA 

2 

NA 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

NA 

126 

NA 

The  three  segments  of  the  EPS  are  in  various 
stages  of  development.  Although  the  EPS 
program's  two  payloads  are  built,  funding 
constraints  resulted  in  reductions  to  the 
requirements  of  the  control  and  planning  segment 
and  gateway  segment,  which  delayed  initial 
operational  capability  by  2  years.  However,  the 
reduction  of  capabilities  also  reduced  the  amount 
of  development  necessary  and  allowed  for  a 
revised  acquisition  strategy  that,  according  to  the 
program  office,  reduced  risk  and  cost. 


Attainment  of  Product  Knowledge 


Projected  as  of  January  2014 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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EPS  Program 

Technology  and  Design  Maturity 

EPS  was  initiated  in  2007  to  fill  a  gap  identified  in 
the  Advanced  Polar  System  study.  The  Under 
Secretary  of  Defense  for  Acquisition,  Technology, 
and  Logistics  directed  the  program  to  proceed  to 
system  development  to  synchronize  the  program's 
payload  schedule  with  the  host  satellite's  production 
timeline.  The  host  program  awarded  the  payload 
development  contract  in  July  2008,  following  an 
acquisition  board  equivalent  to  a  system 
development  decision.  Payload  development 
proceeded  and  all  six  critical  technologies  are  fully 
mature,  according  to  the  program  office.  Both 
payloads  are  built  and  the  first  payload  is  fully 
integrated  into  the  host  satellite,  which  is  currently 
undergoing  system-level  testing. 

In  contrast,  the  control  and  planning  segment 
(CAPS)  and  gateway  development  was  delayed. 
Because  of  funding  constraints,  the  program  office 
used  a  design-to-cost  approach  to  reduce  CAPS 
and  gateway  development  to  the  minimum 
capability  needed.  The  revised  EPS  acquisition 
strategy  incorporating  changes  to  CAPS  and 
gateway  development  was  approved  in  January 
2012.  The  CAPS  design  contract  was  awarded  in 
December  2012,  with  a  planned  system 
development  decision  in  July  2014.  The  gateway 
requires  integration  of  existing  commercial  off-the- 
shelf  and  government  furnished  equipment  and  is 
considered  low  risk  by  the  program  office. 

Other  Program  Issues 

The  reduction  in  capabilities  for  the  CAPS  and 
gateway  allowed  a  change  in  acquisition  strategy 
that  reduced  cost  and  risk  but  also  delayed 
operational  capability  from  fiscal  year  2016  to  2018. 
Under  the  original  acquisition  strategy,  CAPS  was 
planned  as  a  follow-on  increment  to  the  AEHF 
mission  control  segment.  A  sole  source  award  was 
to  be  made  to  AEHF's  prime  contractor  for  CAPS, 
while  the  gateway  was  to  be  competitively  awarded. 
The  reduced  requirements  and  revised  acquisition 
strategy  allowed  for  CAPS  to  be  competitively 
awarded.  The  reduction  of  requirements  nearly 
eliminated  the  development  work  for  the  gateway. 
Consequently,  the  Navy's  Space  and  Naval  Warfare 
Systems  Command  (SPAWAR)  System  Center 
Pacific  will  be  responsible  for  integrating,  testing, 
installing,  certifying,  and  sustaining  the  gateway 


segment.  According  to  the  program  office,  this 
approach  also  reduces  cost  and  schedule  as  it 
would  take  time  for  a  contractor  to  develop  the 
expertise  and  test  beds  that  SPAWAR  System 
Center  Pacific  already  has.  The  reduction  in 
requirements  and  revised  acquisition  strategy 
reduced  overall  program  cost  by  about  $1  billion, 
according  to  program  officials. 

The  payloads  are  expected  to  be  on  orbit  in  fiscal 
years  2015  and  2017  respectively.  By  the  current 
schedule,  CAPS  site  installation  will  occur  in  late 
fiscal  year  2016.  According  to  the  program  office, 
CAPS  is  not  required  for  payload  on-orbit  testing, 
but  it  is  required  for  inter-segment  testing  scheduled 
for  the  first  quarter  of  fiscal  year  2017  and  for  overall 
functioning  of  the  EPS  system.  There  is  adequate 
schedule  margin  according  to  officials,  but  any 
additional  delays  such  as  delayed  funding  or 
program  milestones  will  result  in  a  further  delay  in 
operational  capability. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Evolved  Expendable  Launch  Vehicle  (EELV) 

The  Air  Force's  EELV  program  acquires  launch 
services  to  provide  critical  space  support  to  DOD, 
national  security,  and  other  government  missions. 

The  United  Launch  Alliance  is  currently  the  sole 
provider  of  launch  vehicles  for  U.S.  military  and 
intelligence  satellites.  However,  the  Air  Force  may 
certify  new  launch  providers  to  compete  for  future 
launch  services.  Today,  the  program  consists  of  two 
families  of  launch  vehicles,  support  systems,  and 
launch  site  operations.  We  assessed  the  Atlas  V 
and  the  Delta  IV. 


Source:  ©  2012  United  Launch  Alliance. 


System  development  Production 


▲  ▲  ▲  ▲  ▲ 

Program  start  Development/  Medium-lift  Heavy-lift  Initial 

(12/96)  production  start  first  flight  first  flight  capability 

(6/98)  (8/02)  (12/04)  (12/06) 

▲ 

Milestone 

recertification 

(2/13) 

▲ 

GAO 

review 

(1/14) 

Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  United  Launch 

As  of 

Latest 

Percent 

Services,  LLC 

10/1998 

08/2013 

change 

Program  office:  El  Segundo,  CA 

Research  and  development  cost 

$1,872.6 

$2,460.6 

31.4 

Funding  needed  to  complete: 

Procurement  cost 

$16,403.2 

$61,463.2 

274.7 

R&D:  $81 .4  million 

Total  program  cost 

$18,275.9 

$63,923.7 

249.8 

Procurement:  $42,848.3  million 

Program  unit  cost 

$100,972 

$420,551 

316.5 

Total  funding:  $42,929.7  million 

Total  quantities 

181 

152 

-16.0 

Procurement  quantity:  89 

Acquisition  cycle  time  (months) 

NA 

120 

NA 

We  assessed  EELV  technology  as  mature  with  66 
successful  launches  as  of  December  2013.  We 
could  not  assess  design  or  production  maturity 
using  our  best  practices.  All  of  the  14  EELV 
launch  vehicle  variants  have  launched  at  least 
once.  However,  only  three  variants — the  Atlas  V 
401,  the  Delta  IV  Medium,  and  the  Delta  IV 
Heavy — have  launched  seven  times,  proving 
production  maturity  according  to  an  Aerospace 
Corporation  measure  developed  for  the  program. 
EELV  completed  recertification  after  reporting 
Nunn-McCurdy  critical  unit  cost  breaches  in  the 
March  2012  Selected  Acquisition  Report  and 
transitioned  back  into  the  production  phase  in 
February  2013.  The  program  is  estimated  to  cost 
over  $70  billion  through  2030.  EELV  program 
officials  are  currently  conducting  activities  to 
certify  new  launch  providers  and  signed  a 
contract  modification  to  procure  additional  EELV 
launches. 


Attainment  of  Product  Knowledge 


As  of  January  2014 
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EELV  Program 

Technology,  Design,  and  Production  Maturity 

While  EELV's  technology  is  mature,  we  could  not 
assess  design  and  production  maturity  because  the 
Air  Force  does  not  obtain  information  from  the 
contractor  that  would  facilitate  this  assessment. 
According  to  the  Aerospace  Corporation's 
measurement  known  as  the  "3/7  reliability  rule," 
once  a  vehicle  variant  is  launched  successfully 
three  times  its  design  can  be  considered  mature. 
Similarly,  if  a  vehicle  is  successfully  launched  seven 
times,  both  the  design  and  production  process  can 
be  considered  mature.  Based  on  this  rule,  few 
variants  have  reached  maturity  for  both  design  and 
production.  Each  of  the  9  Atlas  V  variants  and  the 
five  Delta  IV  variants  has  flown  at  least  once, 
demonstrating  the  viability  of  their  technologies,  and 
10  of  the  Atlas  and  Delta  variants  have  flown  three 
times,  demonstrating  design  maturity  according  to 
Aerospace's  rule.  However,  only  3  of  the  14  EELV 
variants  have  demonstrated  production  maturity 
with  seven  flights.  Until  a  launch  vehicle 
configuration  demonstrates  design  and  production 
maturity,  problems  with  fleetwide  designs  or 
production  processes  may  go  undiscovered,  which 
could  cause  significant  cost  and  schedule  risk. 

The  Air  Force  is  investigating  the  root  cause  of  an 
anomaly  with  the  Delta  IV  upper  stage  engine  which 
did  not  perform  as  expected  during  a  launch  in 
October  2012.  According  to  the  EELV  program 
office,  the  initial  phase  of  the  investigation  was 
completed  in  May  2013  but  a  root  cause 
determination  was  not  made.  While  the  engine  has 
performed  normally  on  launches  in  May  and  August 
2013,  the  Air  Force  delayed  a  third  launch  of  the 
Delta  IV  scheduled  for  October  2013,  due  to  new 
conclusions  from  the  investigation  and  to  fully 
understand  the  anomaly  and  reduce  any  potential 
risks. 

Other  Program  Issues 

Due  to  a  program  restructuring  stemming  from  the 
Nunn-McCurdy  breaches  reported  in  the  March 
2012  Selected  Acquisition  Report,  the  program 
developed  a  new  acquisition  program  baseline, 
which  was  approved  in  early  2013.  The  estimated 
costs  for  the  program  are  about  $70  billion  based  on 
the  need  for  151  launches  through  the  year  2030. 
This  estimate  is  around  $35.7  billion  more  than  the 
previous  estimate  reported  in  March  2012.  The 


program  identified  several  causes  for  this  cost 
growth  including  extension  of  the  program  life-cycle 
from  2020  to  2030,  procurement  of  60  additional 
launch  vehicles,  the  inherently  unstable  nature  of 
the  demand  for  launch  services,  and  industrial  base 
instability. 

The  Air  Force  signed  a  contract  modification  in 
December  201 3  to  procure  additional  EELV 
launches  and  has  revised  its  acquisition  strategy  to 
meet  some  of  DOD's  EELV-class  launch  vehicle 
needs.  EELV-class  is  defined  as  a  vehicle  or  family 
of  vehicles  that  can  reliably  launch  a  variety  of 
payloads  up  to  13,500  pounds  to  geosynchronous 
earth  orbit.  The  revised  strategy  outlines  how  to 
certify  a  new  provider's  launch  vehicle  based  on  risk 
classifications  for  individual  payloads.  After  a  new 
provider  completes  its  first  successful  flight,  the  Air 
Force  will  award  early  integration  study  contracts  for 
the  payloads  the  vehicle  is  projected  to  be  capable 
of  launching.  In  November  2012,  the  Air  Force 
awarded  an  EELV-class  launch  service  contract  to 
Space  Exploration  Technologies  (SpaceX),  in  an 
effort  to  support  the  demonstration  of  its  capabilities 
as  part  of  the  certification  process.  In  September 
and  December  2013,  SpaceX  launched  its  first  and 
second  vehicles  toward  certification  respectively. 

Program  Office  Comments 

The  Air  Force  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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Excalibur  Precision  155mm  Projectiles  (Excalibur) 

The  Army's  Excalibur  is  a  family  of  global 
positioning  system-based,  fire-and-forget,  155mm 
cannon  artillery  precision  munitions  intended  to 
provide  improved  range  and  accuracy.  The  near¬ 
vertical  angle  of  fall  is  expected  to  reduce  collateral 
damage,  making  it  more  effective  in  urban 
environments.  The  Army  is  using  an  incremental 
approach  to  deliver  capabilities.  Increment  la-1  is 
fielded,  la-2  is  in  production,  and  lb,  which  is 
expected  to  increase  reliability  and  lower  unit  costs, 
began  production  in  2012.  We  assessed  increment 
lb. 


at 


System  development 


Production 


▲ 

▲ 

▲ 

▲ 

▲ 

▲  ▲ 

Program/ 

Design 

Low-rate 

GAO 

End  operational 

Initial  Full-rate 

development  start 

review 
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test 

capability  decision 

(5/97) 

(4/11) 

(12/12) 

(1/14) 

(3/14) 

(7/14)  (7/14) 

Program  Essentials 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Prime  contractor:  Raytheon 
Program  office:  Picatinny  Arsenal,  NJ 
Funding  needed  to  complete: 

R&D:  $7.9  million 
Procurement:  $75.9  million 
Total  funding:  $83.9  million 
Procurement  quantity:  952 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

02/2003 

08/2013 

change 

$805.1 

$1,137.6 

41.3 

$4,218.4 

$702.0 

-83.4 

$5,023.5 

$1,839.6 

-63.4 

$0,066 

$0,281 

328.4 

76,677 

6,555 

-91.5 

NA 

206 

NA 

Note:  Total  quantities  include  2,849  increment  lb  projectiles. 


The  Excalibur  program  has  made  multiple  design 
changes  to  address  reliability  concerns  since 
2012,  including  a  risk  that  it  may  not  meet  its 
reliability  threshold  requirement.  The  lb  round  still 
does  not  meet  insensitive  munitions 
requirements.  According  to  program  officials, 
however,  it  does  meet  military  standards  and  the 
program  has  projected  test  dates  to  address 
concerns  of  the  Army  Insensitive  Munitions 
Board.  The  program  reduced  the  number  of 
rounds  planned  for  procurement  due  to  a 
recommendation  in  the  conference  report 
accompanying  the  Consolidated  and  Further 
Continuing  Appropriations  Act,  2013  and 
sequestration  but  has  requested  restitution  of 
those  funds  in  fiscal  years  2015  and  2016. 


Attainment  of  Product  Knowledge 

As  of  January  2014 

Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

• 

•  Demonstrate  all  critical  technologies  in  a  realistic 
environment 

• 

•  Complete  preliminary  design  review 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

• 

•  Test  a  system-level  integrated  prototype 

• 

Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

• 

^  Knowledge  attained  ■  ■  ■  =  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Excalibur  Program 

Technology  and  Design  Maturity 

The  Excalibur  program's  critical  technology  is  fully 
mature.  The  program  has  made  multiple 
incremental  design  changes  intended  to  increase 
reliability  since  the  low-rate  production  decision  in 
2012.  For  example,  the  contractor  producing  the 
Global  Positioning  System  (GPS)  receiver  made 
hardware  and  software  changes  to  increase 
receiver  reliability  in  the  gun  launch  environment. 
Additionally,  the  program  has  made  changes  to  the 
guidance  assembly  and  the  fuze  safe  and  arm 
device  to  address  failures  during  lb  round 
developmental  testing.  There  have  been  no  design 
changes  to  the  warhead  or  the  operational  flight 
software.  The  lb  round  still  does  not  meet 
requirements  related  to  insensitive  munitions — 
ensuring  that  a  round  will  not  detonate  under  any 
condition  other  than  its  intended  mission.  According 
to  program  officials,  however,  it  does  meet  military 
standards.  The  program  reports  that  a  follow-up  test 
to  meet  the  Army  Insensitive  Munitions  Board 
requirements,  which  was  delayed  due  to 
sequestration  and  furloughs  during  the  government 
shutdown,  is  projected  for  early  2014.  Ensuring 
compliance  with  this  requirement  could  require 
design  changes  and  the  program  is  uncertain  of  the 
total  number  of  drawings  that  will  ultimately  be 
required. 

The  program  reported  that  it  received  a  funding 
increase  in  fiscal  year  2014  of  $5  million  to  support 
the  completion  of  initial  operational  test  and 
evaluation,  which  was  subsequently  reduced  due  to 
sequestration  for  a  net  increase  of  $4.3  million.  All 
performance  and  safety  requirements  have  been 
met,  but  reliability  testing  will  not  be  complete  until 
after  production  has  begun. 

Production  Maturity 

According  to  program  officials,  the  program  is  at  low 
risk  of  not  meeting  its  reliability  threshold  of  90 
percent  and  at  moderate-to-high  risk  of  not  meeting 
its  reliability  objective.  However,  given  test  results 
since  the  low-rate  decision,  the  program  expects  to 
meet  the  threshold  reliability  requirement.  The 
program  has  been  authorized  to  procure  1800 
rounds  in  low-rate  production  and  will  seek 
authorization  to  procure  the  remaining  rounds  at  the 
full-rate  production  decision,  tentatively  scheduled 
for  mid-2014. 


Other  Program  Issues 

The  program  is  still  projecting  to  produce  3,455  lb 
rounds,  but  its  fiscal  year  2013  procurement  funding 
was  cut  by  $47.5  million  due  to  a  recommendation 
in  the  conference  report  accompanying  the 
Consolidated  and  Further  Continuing 
Appropriations  Act,  2013  and  sequestration.  As  a 
result  the  program  is  reporting  a  17  percent 
reduction  of  rounds  for  procurement.  The  program, 
however,  has  requested  restoration  of  funding  for 
those  rounds  in  fiscal  years  2015  and  2016.  The 
project  originally  planned  for  funding  to  end  in  fiscal 
year  2014;  spreading  funding  into  future  years  will 
increase  the  life  of  the  program  and  may  have 
additional  costs. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  stated  that  Excalibur  continues  to 
provide  commanders  with  an  organic  precision 
targeting  capability  resulting  in  over  700  rounds 
being  fired  in  combat  to  date.  They  also  stated  that 
the  program  continues  to  execute  within  all 
acquisition  program  baseline  parameters  and  is 
meeting  or  exceeding  all  five  of  its  key  performance 
parameters.  Excalibur  Increment  lb  has  shown 
performance  improvements  and  continues  to 
execute  to  the  plan  approved  by  both  the  Army  and 
joint  insensitive  munitions  board  and  has  seen 
improvements  in  the  area  of  less  sensitive 
reactions. 

Program  officials  stated  that  the  final  Excalibur 
increment  began  low-rate  initial  production  in  first 
quarter  fiscal  year  2014  and  successfully  completed 
first  article  testing  in  December  2013.  The  current 
Excalibur  production  contract  contains  fixed  price 
procurement  options  to  ensure  consistent 
procurement  costs  through  fiscal  year  2016.  The 
program  also  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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F-22  Increment  3.2B 

The  Air  Force's  F-22  Raptor  is  a  stealthy  air-to-air 
and  air-to-ground  fighter/attack  aircraft.  The  Air 
Force  established  an  F-22  modernization  and 
improvement  program  in  2003  to  add  enhanced  air- 
to-ground,  information  warfare,  reconnaissance, 
and  other  capabilities  and  to  improve  the  reliability 
and  maintainability  of  the  aircraft.  Increment  3.2B, 
the  fourth  increment  of  the  modernization  program, 
was  initially  managed  as  part  of  the  F-22  baseline 
program,  but  is  now  being  managed  as  a  separate 
major  defense  acquisition  program. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Wright-Patterson  AFB, 

06/2013 

09/2013 

change 

OH 

Research  and  development  cost 

$1,222.1 

$1,218.8 

-0.3 

Funding  needed  to  complete: 

Procurement  cost 

$344.6 

$339.1 

-1.6 

R&D:  $543.1  million 

Total  program  cost 

$1,566.7 

$1,557.9 

-0.6 

Procurement:  $339.1  million 

Program  unit  cost 

$10,308 

$10,249 

-0.6 

Total  funding:  $882.2  million 

Total  quantities 

152 

152 

0.0 

Procurement  quantity:  143 

Acquisition  cycle  time  (months) 

72 

75 

4.2 

Increment  3.2B  is  an  enhancement  to  the  F-22, 
bringing  upgraded  electronic  protection, 
geolocation,  and  intra-flight  data  link  capabilities, 
and  integrating  AIM-9X  and  AIM-120D  missiles. 
The  Air  Force  received  approval  to  begin  system 
development  of  Increment  3.2B  as  a  separate 
major  defense  acquisition  program  in  June  2013. 
The  one  reported  critical  technology  has  been 
demonstrated  in  a  relevant,  but  not  a  realistic, 
environment.  Full  maturity  is  expected  by  the 
critical  design  review  in  October  2015,  by  which 
point  the  Air  Force  also  anticipates  achieving  full 
design  stability.  There  may  be  a  production  gap  of 
two  to  four  years  between  qualification  of 
hardware  for  the  program  and  the  beginning  of 
low-rate  initial  production.  The  program  office 
stated  that  it  has  mitigated  risk  related  to  AIM-9X 
software  development. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

Q  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  F-22  Inc  3.2B 


F-22  Inc  3.2B  Program 

Technology  and  Design  Maturity 

Increment  3.2B's  sole,  identified  critical  technology, 
a  geolocation  algorithm,  is  not  yet  fully  mature 
having  been  demonstrated  only  in  a  relevant 
environment.  The  program  office  expects  the 
technology  to  reach  full  maturity  by  October  2015. 
The  Air  Force  also  anticipates  achieving  full  design 
stability  by  the  October  2015  critical  design  review 
as  92  percent  of  system  level  drawings  are  currently 
releasable.  A  knowledge-based  acquisition 
approach  is  a  cumulative  process,  and  mature 
technology  is  a  prerequisite  for  moving  into  system 
development.  The  program  intends  to  demonstrate 
that  the  design  can  perform  as  expected  by 
integrating  and  testing  all  key  subsystems  and 
components  prior  to  the  critical  design  review. 

Production  Maturity 

Prior  to  system  development,  DOD  systems 
engineering  personnel  identified  a  production  gap  of 
two  to  four  years  between  qualification  of  hardware 
for  Increment  3.2B  and  the  beginning  of  low-rate 
initial  production  as  a  risk  as  some  of  the  electronic 
components  used  may  no  longer  be  available  when 
production  begins.  The  program  office  stated  that  it 
is  working  closely  with  suppliers  to  mitigate 
identified  risks  through  an  advance  procurement 
contract.  Program  officials  partly  attributed  the  gap 
to  an  early  start  made  on  hardware  development 
given  initial  assumptions  about  the  amount  of 
development  involved.  Additionally,  the  Defense 
Contract  Management  Agency  has  noted  supplier 
developmental  issues  with  displays  and  processing 
modules  which  support  Increment  3.2B. 

Other  Program  Issues 

Consistent  with  direction  from  Congress  and  the 
Office  of  the  Secretary  of  Defense,  the  Air  Force 
designated  F-22  Increment  3.2B  as  a  distinct  major 
defense  acquisition  program  rather  than  managing 
it  as  part  of  the  F-22  baseline  program.  About  half  of 
the  program's  $1.2  billion  estimated  development 
cost  was  spent  under  the  F-22  baseline  program. 
Originally  scheduled  for  December  2012,  the 
approval  to  formally  initiate  the  Increment  3.2B 
acquisition  program  and  begin  system  development 
was  delayed  until  June  201 3  as  it  took  longer  than 
expected  to  prepare  Air  Force  and  independent  cost 
estimates.  According  to  cost  estimators,  the  delay  is 
attributable  to  their  having  to  reconstruct  missing 


cost  information  from  prior  modernization 
increments  in  order  to  estimate  costs  for  Increment 
3.2B. 

An  initial  risk  assessment  noted  that  if  the  Increment 
3.2B  program  did  not  receive  the  customized 
software  that  allows  the  F-22  to  use  the  AIM-9X  air- 
to-air  missile  in  time  for  operational  testing,  it  might 
face  a  delay  in  its  overall  fielding  schedule.  The  F- 
22  program  office  stated  that  it  has  mitigated  risk 
related  to  missile  software  development,  and  has 
signed  an  agreement  with  the  AIM-9X  program 
office  on  technical  and  schedule  requirements  for 
integrating  the  missile.  The  F-22  program  office 
plans  to  field  limited  AIM-9X  capabilities  prior  to  the 
completion  of  Increment  3.2B  by  including  them  as 
part  of  an  earlier  software  update. 

Program  Office  Comments 

The  Air  Force  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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F-35  Lightning  II  Program  (F-35) 


DOD's  F-35  program  is  developing  a  family  of 
stealthy,  strike  fighter  aircraft  for  the  Navy,  Air 
Force,  Marine  Corps,  and  U.S.  allies,  with  the  goal 
of  maximizing  commonality  to  minimize  life-cycle 
costs.  The  carrier-suitable  variant  will  complement 
the  Navy  F/A-18E/F.  The  Air  Force  variant  is 
expected  to  replace  the  air-to-ground  attack 
capabilities  of  the  F-16  and  A-10,  and  complement 
the  F-22A.  The  short  take-off  and  vertical  landing 
(STOVL)  variant  is  expected  to  replace  the  Marine 
Corps  F/A-18  and  AV-8B  aircraft. 


Source:  Courtesy  of  Lockheed  Martin. 


oncept 


System  development 


Production 


▲  ▲  ▲▲▲▲▲▲▲▲ 

Program  Development  Design  Production  Milestone  GAO  Start  USMC  initial  USAF  initial  USN  initial 

start  start  review  decision  recertification  review  operational  test  capability  capability  capability 

(11/96)  (10/01)  (2/06  &  6/07)  (6/07)  (3/12)  (1/14)  (6/15)  (7/15)  (8/16)  (8/18) 


Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin, 

As  of 

Latest 

Percent 

Pratt  and  Whitney 

10/2001 

08/2013 

change 

Program  office:  Arlington,  VA 

Research  and  development  cost 

$40,995.1 

$61,795.8 

50.7 

Funding  needed  to  complete: 

Procurement  cost 

$181,875.9 

$266,469.1 

46.5 

R&D:  $5,516.9  million 

Total  program  cost 

$224,774.7 

$332,320.0 

47.8 

Procurement:  $231,243.5  million 

Program  unit  cost 

$78,428 

$135,254 

72.5 

Total  funding:  $239,559.1  million 

Total  quantities 

2,866 

2,457 

-14.3 

Procurement  quantity:  2,293 

Acquisition  cycle  time  (months) 

116 

237 

104.3 

Although  the  F-35  program  has  invested  billions 
of  dollars  and  ordered  150  production  aircraft, 
gaps  in  product  knowledge  persist.  Four  critical 
technologies  are  not  mature  and  the  program 
continues  to  experience  design  changes. 
Development  testing  is  progressing  but  is  far  from 
complete  and  will  likely  drive  more  changes  in 
design  and  manufacturing  processes.  While 
manufacturing  efforts  are  improving,  only  twenty- 
five  percent  of  the  program's  critical 
manufacturing  processes  are  mature  and  capable 
of  consistently  producing  quality  parts,  and  fewer 
planes  were  produced  in  2013  than  initially 
planned.  Software  development  has  progressed; 
but  if  additional  delays  are  encountered  initial 
operating  capability  dates  could  be  at  risk.  Long¬ 
term  affordability  and  sustainment  costs  remain 
challenging. 


Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 

environment  LJ 

•  Demonstrate  all  critical  technologies  in  a  realistic  s~\ 

environment 

•  Complete  preliminary  design  review  % 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  % 

•  Test  a  system-level  integrated  prototype  % 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control  O 

•  Demonstrate  critical  processes  on  a  pilot  production  line  0 

•  Test  a  production-representative  prototype  £ 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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F-35  Program 

Technology  Maturity 

Three  of  the  program's  critical  technologies  are 
nearing  maturity  while  one  is  immature.  The 
program  has  made  progress  addressing  technical 
risks  including  the  helmet  mounted  display,  which  is 
needed  for  the  aircraft  to  fully  perform  its  missions, 
and  the  Autonomic  Logistics  Information  System 
(ALIS),  which  is  critical  to  the  operations  and 
sustainment  of  the  fleet.  The  program  recently 
chose  to  end  development  of  an  alternate  helmet 
due  to  progress  made  on  the  original  helmet  design 
and  work  on  development  of  a  newer  generation 
helmet.  ALIS  integration  and  testing  continues  with 
the  next,  more  capable  version  expected  to  be 
delivered  in  2014. 

Design  Maturity 

Although  the  three  aircraft  designs  were  not  stable 
at  critical  design  reviews  in  2006  and  2007,  all 
expected  engineering  drawings  have  since  been 
released.  However,  issues  discovered  in  testing 
continue  to  drive  design  changes.  In  2013,  bulkhead 
and  rib  cracks  were  discovered  in  the  STOVL 
variant  that  may  require  the  program  to  make 
changes  to  production  and  retrofit  aircraft.  The 
carrier  variant's  new  arresting  hook  system 
successfully  completed  a  critical  design  review  and 
is  expected  to  be  ready  for  carrier  use  by  October 
2014.  With  significant  testing  remaining,  additional 
design  changes  are  likely. 

Production  Maturity 

DOD  has  invested  about  $35  billion  in  procuring  150 
aircraft  through  2013.  The  contractor  uses  statistical 
process  controls  as  one  means  to  assess  critical 
manufacturing  processes.  Twenty  five  percent  of 
those  processes  are  currently  judged  capable  of 
consistently  producing  quality  parts  at  the  best 
practice  standards.  Production  efforts  have 
improved  as  the  production  line  continues  to  show 
efficiencies  and  quality  metrics  show  positive 
trends.  However,  in  2013,  problems  with  engine 
hoses,  engine  turbines,  and  specialty  metals  halted 
production  deliveries  for  three  months.  In  2013,  35 
planes  were  delivered  to  the  government — eight 
less  than  originally  planned.  Aircraft  deliveries  were 
postponed  for  one  month  while  the  runway  at  the 
final  assembly  facility  was  resurfaced. 


Other  Program  Issues 

In  2013,  the  military  services  re-established  initial 
operating  capability  dates  for  their  respective  F-35 
variants.  To  achieve  those  dates,  the  Marine  Corps 
and  Air  Force  will  accept  basic  warfighting 
capabilities  provided  by  software  Blocks  2B  and  31 
respectively,  while  the  Navy  intends  to  wait  for  the 
full  suite  of  capabilities  provided  by  software  Block 
3F.  Block  2B  development  is  currently  behind 
schedule,  which  could  impact  the  development  of 
Block  31,  and  subsequently  3F.  Any  additional 
software  delays  could  put  initial  operating  capability 
dates  at  risk  as  software  is  essential  to  achieve 
aircraft  capabilities  such  as  sensor  fusion,  weapons 
and  fire  control,  maintenance  diagnostics,  and 
propulsion.  In  addition,  the  program's  long-term 
affordability  remains  a  challenge.  The  amount  of 
annual  funding  required  to  complete  development 
and  procurement  of  the  F-35  is  more  than  $12 
billion  on  average,  while  the  cost  to  operate  and 
support  the  aircraft  is  estimated  at  more  than  $1 
trillion  over  its  life-cycle.  These  estimates  may 
change  over  time  as  testing  progresses  and  more 
concrete  information  becomes  available. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  noted  that  it  appreciates  GAO's  reviews  in 
assisting  the  program  by  identifying  areas  for 
improvement.  The  program  has  made  great  strides 
in  the  past  year  to  further  stabilize  technology, 
design,  and  production  maturity,  and  work  continues 
on  identified  areas  to  improve  program 
performance.  The  program  ended  development  of 
the  alternate  F-35  helmet  as  further  testing 
indicated  it  is  acceptable  for  USMC  initial  operating 
capability.  Continued  improvements  will  be  made  in 
the  Gen  III  helmet.  This  decision  includes  a 
guarantee  from  industry  to  reduce  the  unit  cost  by 
12  percent  from  previous  estimates.  Aircraft  and 
engine  quality  continues  to  improve  as 
manufacturing  processes  mature,  and  unit  costs 
continue  to  come  down  lot  over  lot.  The  program 
has  taken  steps  to  improve  management  of 
concurrency  risk  and  minimize  costs  to  the  services 
by  reducing  the  time  required  to  implement  changes 
into  the  production  line  so  that  fewer  aircraft  need 
post-production  retrofits. 
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Family  of  Beyond  Line-of-Sight  -  Terminals  (FAB-T) 

The  Air  Force's  FAB-T  program  plans  to  provide  a 
family  of  satellite  communications  terminals  for 
airborne  and  ground-based  users  to  replace  many 
program-unique  terminals.  It  is  being  designed  to 
work  with  current  and  future  communications 
capabilities  and  technologies.  FAB-T  is  expected  to 
provide  voice  and  data  over  military  satellite 
communications  for  nuclear  and  conventional  forces 
through  ground  command  posts  and  E-6  and  E-4 
aircraft,  and  was  originally  planned  to  provide  force 
element  capabilities  on  B-2,  B-52,  and  RC-135 
aircraft. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Prime  contractor:  Boeing,  Raytheon 
Program  office:  Bedford,  MA 
Funding  needed  to  complete: 

R&D:  $196.0  million 
Procurement:  $2,190.6  million 
Total  funding:  $2,386.6  million 
Procurement  quantity:  216 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 

As  of 
12/2006 

$1,616.7 

Latest 

10/2013 

$2,406.0 

Percent 

change 

48.8 

Procurement  cost 

$1,736.9 

$2,212.0 

27.4 

Total  program  cost 

$3,353.6 

$4,618.0 

37.7 

Program  unit  cost 

$15,526 

$18,772 

20.9 

Total  quantities 

216 

246 

13.9 

Acquisition  cycle  time  (months) 

129 

204 

58.1 

The  latest  cost  data  do  not  reflect  the  current  cost  of  the  program  and  the  latest  quantities  do  not 
reflect  recent  program  decisions.  A  new  acquisition  program  baseline  has  not  yet  been  approved. 


The  program  is  approaching  a  production 
decision  with  a  stable,  demonstrated  design  but 
without  having  conducted  key  assessments.  Due 
to  continued  cost  and  schedule  growth  in 
developing  this  design,  the  Air  Force  signed  a 
development  contract  with  an  additional  vendor  in 
September  2012  to  establish  an  alternate  source 
for  a  system  with  capabilities  similar  to  the  original 
vendor's  FAB-T  effort.  An  acquisition  baseline 
updated  to  reflect  the  award  to  a  second  vendor, 
and  other  changes  to  the  program  strategy  since 
2010,  has  been  delayed  and  will  not  be  approved 
until  after  the  Air  Force  determines  which  vendor 
will  move  forward  into  production.  Despite 
numerous  changes  to  the  program,  an 
independent  cost  estimate  has  not  been 
conducted  since  2009,  but  is  expected  in  early 
2014. 
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Common  Name:  FAB-T 


FAB-T  Program 

Technology  Maturity 

According  to  the  program  office,  all  of  the  critical 
technologies  for  both  vendors  are  fully  mature 
based  on  the  vendors'  self-assessments.  The 
program  does  not  currently  plan  to  conduct  an 
independent  technology  readiness  assessment  for 
the  alternate  vendor.  The  program  office  also  stated 
that  both  vendors  have  functional  prototypes  being 
utilized  for  testing. 

Design  Maturity 

According  to  the  program  office,  FAB-T's  design  is 
stable  for  both  the  original  and  alternate  vendor 
development  programs.  The  original  vendor 
completed  functional  qualification  testing  in  August 
2013.  The  alternate  vendor  completed  a  system- 
level  design  review  in  June  2013  and  demonstrated 
a  system-level  integrated  prototype  in  October 
2013. 

Production  Maturity 

In  2012,  FAB-T's  acquisition  strategy  was  changed 
to  initiate  two  possible  production  paths:  one 
providing  both  command  and  force  element 
terminals,  and  the  other  only  command  terminals. 
Both  vendors  submitted  proposals  to  compete  for 
both  approaches.  In  December  2013,  the  Air  Force 
chose  to  pursue  the  command  terminal  only 
approach  and  the  program  plans  to  make  a  down- 
select  decision  to  one  vendor  in  the  second  quarter 
of  fiscal  year  2014.  Depending  on  which  vendor  is 
chosen,  the  low-rate  production  decision  is  planned 
for  either  March  2014  or  September  2014. 

A  2012  acquisition  decision  memorandum  called  for 
several  steps  before  the  award  of  FAB-T's 
production  contracts,  including:  an  independent  cost 
estimate;  a  revised  acquisition  program  baseline; 
and  a  defense  acquisition  board  review.  Program 
officials  stated  that  neither  the  cost  estimate  nor  the 
revised  baseline  will  be  completed  prior  to  the 
down-select  decision.  Department  officials  stated 
that  the  review  has  been  delayed  and  not  yet 
rescheduled.  Also,  FAB-T  does  not  plan  to 
independently  assess  manufacturing  readiness 
levels  prior  to  the  down-select  decision.  Air  Force 
decisions  to  commit  to  a  production  contractor 
before  the  program  is  fully  assessed  introduce 
additional  acquisition  risk. 


Other  Program  Issues 

In  October  201 0,  an  independent  review  team  noted 
that  the  program  was  likely  in  breach  of  critical 
Nunn-McCurdy  unit  cost  thresholds  but  a  breach 
has  not  yet  been  reported  by  the  program.  In  2012, 
the  FAB-T  program  made  several  revisions  to  its 
acquisition  strategy — including  modification  of  the 
original  development  contract  from  cost  plus  award 
fee  to  firm  fixed-price  and  the  addition  of 
competition  by  awarding  the  alternate  vendor 
contract  in  response  to  these  concerns  about  cost 
growth.  However,  a  new  acquisition  program 
baseline  and  independent  cost  estimate  that  reflect 
these  changes  and  their  projected  costs  will  not  be 
completed  until  2014. 

According  to  Department  officials,  the  requirement 
for  the  force  element  terminals  remains  in  the 
program  but  is  not  currently  funded.  If  not  integrated 
with  the  B-2  and  B-52  bomber  platforms,  FAB-T 
may  not  meet  its  full  range  of  planned 
communications  capabilities  as  some  are  based  on 
the  interaction  of  bomber  aircraft  with  intelligence, 
surveillance,  and  reconnaissance  aircraft  and 
command  terminals. 

Program  Office  Comments 

According  to  program  officials,  the  FAB-T  program 
office  continued  to  execute  the  revised  acquisition 
strategy  in  2013.  Significant  progress  was  made  by 
both  contractors  on  their  respective  development 
contracts.  With  respect  to  production  efforts, 
production  planning  contracts  were  awarded  to  lock 
down  prices.  In  addition,  the  competition  for 
command  terminals  is  on-going,  and  the  Air  Force 
has  scheduled  a  down  select  decision  in  the  second 
quarter  of  fiscal  year  2014.  The  independent  cost 
estimate  and  acquisition  program  baseline  will  then 
be  updated  to  re-baseline  the  program.  To  address 
the  cost  thresholds  and  breach  concerns,  program 
officials  said  they  are  confident  that  adequate 
technology  and  manufacturing  assessments,  along 
with  the  revised  cost  and  schedule  evaluation,  will 
be  conducted  prior  to  the  decision  to  exercise  the 
first  production  option.  The  program  office's 
assessment  is  that  there  will  be  a  stable  and  mature 
design,  with  a  reliable  manufacturing  process,  to 
proceed  into  operational  unit  production  for  this 
critical  operational  capability. 
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Common  Name:  CVN  78 


Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 


The  Navy's  CVN  78  class  of  nuclear-powered 
aircraft  carriers  is  designed  to  improve  operational 
efficiency,  enable  higher  sortie  rates,  and  reduce 
manpower  through  the  use  of  advanced  propulsion, 
aircraft  launch  and  recovery,  and  survivability 
technologies.  The  lead  ship,  CVN  78,  was  70 
percent  complete  at  launch;  however,  its  delivery  is 
delayed  until  March  2016.  Also,  contract 
negotiations  prompted  the  Navy  to  defer  the 
construction  contract  for  CVN  79  until  early  fiscal 
year  2015. 


Source:  U.S.  Navy. 
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Program  Essentials 

Prime  contractor:  Huntington  Ingalls 
Industries 

Program  office:  Washington,  DC 
Funding  needed  to  complete: 

R&D:  $533.6  million 
Procurement:  $16,971.8  million 
Total  funding:  $17,508.7  million 
Procurement  quantity:  1 


Program  Performance  (fi 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


year  2014  dollars  in  millions) 


As  of 

Latest 

Percent 

04/2004 

08/2013 

change 

$5,051.9 

$4,867.4 

-3.7 

$32,363.3 

$31,081.1 

-4.0 

$37,415.2 

$36,008.1 

-3.8 

$12,471,733 

$12,002,694 

-3.8 

3 

3 

0.0 

137 

155 

13.1 

The  Navy  awarded  a  construction  contract  for 
CVN  78  in  September  2008  when  only  5  of  the 
ship's  13  current  critical  technologies  were  mature 
and  the  ship's  three-dimensional  (3D)  product 
model  was  incomplete.  Since  then,  the  lead  ship's 
procurement  costs  have  grown  by  over  22 
percent,  from  $10.5  billion  to  $12.8  billion.  At 
present,  7  of  the  currently  planned  technologies 
are  mature,  with  the  rest  approaching  maturity, 
and  the  ship's  3D  product  model  is  complete. 
However,  maintaining  design  stability  depends  on 
currently  immature  technologies  fitting  within  the 
space,  weight,  cooling,  and  power  reservations 
allotted  them  within  the  ship.  Construction  to  date 
has  been  impeded  by  critical  technology  system 
delays,  material  shortages,  and  engineering 
challenges. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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CVN  78  Program 

Technology  Maturity 

According  to  the  Navy,  7  of  the  13  critical 
technologies  for  CVN  78  are  mature,  and  the 
remaining  6  are  approaching  maturity.  To  meet 
required  installation  dates  aboard  CVN  78,  the  Navy 
produced  several  of  these  technologies,  such  as  the 
volume  search  radar  (VSR),  prior  to  demonstrating 
their  maturity — a  strategy  GAO's  prior  work  has 
shown  introduces  risk  of  late  and  costly  design 
changes  and  rework.  The  VSR  is  a  component  of 
the  dual  band  radar  (DBR),  which  has  been 
delivered  to  CVN  78,  and  is  undergoing  design 
modifications  needed  to  complete  shipboard 
integration.  According  to  the  Navy,  testing  in  the 
spring  of  2015  will  show  whether  these 
modifications  were  successful  or  a  more  extensive 
redesign  of  the  system  is  required,  which  could 
delay  DBR  deliveries  by  up  to  4  years.  Deficiencies 
affecting  water  twister  components — used  to  absorb 
energy  when  arresting  aircraft — of  the  advanced 
arresting  gear  (AAG)  technology  continue  to  disrupt 
the  system's  development.  Recent  water  twister 
redesign  proved  unsuccessful  in  testing  last  year. 
The  Navy  resolved  problems  with  the  redesign  and 
is  planning  for  concurrent  testing.  Despite  these 
steps,  the  Navy  forecasts  AAG  land-based  testing 
to  be  complete  in  August  2016 — a  new  delay  of 
nearly  two  years — and  after  the  Navy  has  accepted 
CVN  78  delivery.  The  electromagnetic  aircraft 
launch  system  (EMALS)  has  successfully  launched 
aircraft  during  land-based  testing  using  a  single 
launcher  and  four  motor  generators.  The  shipboard 
system  will  employ  a  more  complex  configuration 
with  more  launchers  and  generators  sharing  a 
power  interface. 

Design  Maturity 

CVN  78  completed  its  3D  product  model  in 
November  2009 — over  a  year  after  the  construction 
contract  award.  While  the  model  is  now  considered 
functionally  complete,  maintaining  design  stability 
depends  on  technologies  fitting  within  the  space, 
weight,  cooling,  and  power  reservations  allotted 
them.  Shipboard  testing  may  reveal  a  need  for 
design  changes.  Also,  as  construction  progresses, 
the  shipbuilder  is  discovering  "first-of-class"  type 
design  changes,  which  it  is  using  to  update  the 
model  prior  to  CVN  79  construction. 


Production  Maturity 

According  to  program  officials,  CVN  78  is 
approximately  70  percent  complete.  Lead  ship 
procurement  costs  for  the  lead  ship  have  grown  by 
over  22  percent  since  construction  authorization  in 
fiscal  year  2008  due  in  part  to  problems 
encountered  in  construction.  Out-of-sequence  work 
driven  largely  by  material  shortfalls,  engineering 
challenges,  and  delays  developing  and  installing 
certain  critical  technologies  the  Navy  provides  to  the 
shipbuilder  for  installation  has  affected  construction 
progress. 

Other  Program  Issues 

The  Navy  deferred  award  of  the  CVN  79  detail 
design  and  construction  contract  from  late  fiscal 
year  201 3  to  the  first  quarter  of  fiscal  year  2015. 
According  to  the  Navy,  continuing  contract 
negotiations  provide  an  opportunity  to  incorporate 
process  improvements  into  construction  plans.  The 
Navy  has  undertaken  an  in-depth  review  of  CVN  79 
requirements  and  capabilities  to  identify  cost  trades, 
which  it  hopes  can  facilitate  an  agreement  on 
contract  terms.  These  actions  are  consistent  with 
recommendations  we  made  in  September  2013  to 
defer  the  CVN  79  construction  contract  and  to 
conduct  a  cost-benefit  analysis  on  Ford-class 
capability  requirements  and  the  time  and  money 
needed  to  field  systems  to  provide  these 
capabilities. 

Program  Office  Comments 

According  to  the  program  office,  CVN  78  displaced 
77,000  tons  and  was  70  percent  complete  at 
launch — the  highest  levels  achieved  in  aircraft 
carrier  new  construction.  The  program  office  also 
reported  that  labor  inefficiencies  during  ship 
erection  are  past  and  the  principal  risk  remaining  is 
in  shipboard  testing.  Concerns  over  system 
integration  within  platform  space,  weight,  and  power 
reservations  have  been  resolved.  Land  based 
testing  for  EMALS  and  DBR  has  progressed 
enough  that  program  officials  do  not  anticipate 
significant  redesign.  Further,  the  AAG  test  schedule 
remains  on  track  to  support  ship  delivery  and  sea 
trials.  Lastly,  the  Navy  plans  to  modify  the  CVN  79 
construction  preparation  contract  to  extend  the 
terms  of  the  contract  and  avoid  a  production  break 
during  negotiations  on  the  detail  design  and 
construction  contract  without  delaying  ship  delivery. 
Program  officials  also  provided  technical  comments 
that  were  incorporated  where  deemed  appropriate. 
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Global  Positioning  System  III  (GPS  III) 

The  Air  Force's  Global  Positioning  System  III  (GPS 
III)  program  plans  to  develop  and  field  a  new 
generation  of  satellites  to  supplement  and 
eventually  replace  GPS  satellites  currently  in  use. 

Other  programs  are  developing  the  ground  system 
and  user  equipment.  GPS  III  is  to  be  developed 
incrementally.  We  assessed  the  first  increment, 
which  intends  to  provide  capabilities  such  as  a 
stronger  military  navigation  signal  to  improve 
jamming  resistance  and  a  new  civilian  signal  that 
will  be  interoperable  with  foreign  satellite  navigation 
systems. 


Source:  U.S.  Air  Force. 
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Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Program  Essentials 

Prime  contractor:  Lockheed  Martin 
Program  office:  El  Segundo,  CA 
Funding  needed  to  complete: 

R&D:  $261.2  million 
Procurement:  $495.4  million 
Total  funding:  $756.6  million 
Procurement  quantity:  2 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

05/2008 

11/2013 

change 

$2,654.8 

$2,795.0 

5.3 

$1,490.5 

$1,581.5 

6.1 

$4,145.4 

$4,376.5 

5.6 

$518,169 

$547,059 

5.6 

8 

8 

0.0 

NA 

NA 

NA 

We  could  not  calculate  acquisition  cycle  times  for  the  first  increment  of  the  GPS  III  program  because 
initial  operational  capability  will  not  occur  until  satellites  from  a  future  increment  are  fielded. 


The  GPS  III  program's  eight  critical  technologies 
have  been  assessed  as  mature,  but  three  have 
yet  to  complete  qualification  testing.  The  program 
reported  that  its  design  and  production  processes 
are  currently  mature  and  a  prototype  was 
constructed  to  reduce  risk.  However,  design 
changes  have  been  required  due  to  issues 
identified  in  testing.  Late  hardware  delivery 
resulted  in  delayed  testing  of  the  prototype.  The 
GPS  III  program  delayed  the  first  space  vehicle's 
availability  for  launch  from  April  2014  to  April 
2016 — a  24  month  delay — due  to  late  hardware 
deliveries  and  technical  problems  identified  in 
testing. 


Attainment  of  Product  Knowledge 
As  of  January  2014 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  w 

•  Demonstrate  all  critical  technologies  in  a  realistic 
environment 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  £ 

•  Test  a  system-level  integrated  prototype  £ 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control  £ 

•  Demonstrate  critical  processes  on  a  pilot  production  line  £ 

•  Test  a  production-representative  prototype  o 

0  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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GPS  III  Program 

Technology  and  Design  Maturity 

The  program  office  reports  that  all  eight  of  its  critical 
technologies  are  mature  and  that  its  design  is 
stable.  Three  of  the  GPS  III  critical  technologies, 
including  the  timekeeping  system  and  the  key  GPS 
signal  generator,  have  not  yet  completed 
qualification  testing,  which  is  behind  schedule. 
Based  on  the  number  of  design  drawings  released 
to  manufacturing,  the  design  has  been  mature  since 
the  program's  2010  design  review.  However,  design 
changes  have  been  required  due  to  problems 
identified  in  testing  of  key  components.  The 
program  office  reported  that  it  has  demonstrated 
that  the  design  can  meet  requirements  by  testing  a 
system-level  integrated  prototype  that  includes  all 
key  subsystems  and  components  just  with  less 
redundancy  in  the  GPS  III  satellites. 

Production  Maturity 

At  the  time  of  GPS  Ill's  production  decision,  the 
program  office  reported  a  level  of  manufacturing 
process  maturity  that  indicated  its  processes  were 
in  control  and  production  could  begin.  However,  a 
complete  GPS  III  satellite  was  not  tested  prior  to  the 
production  decision  nor  has  one  been  tested  to 
date.  To  mitigate  risk,  the  program  developed  a 
prototype  to  prove  out  production  processes  prior  to 
integrating  and  testing  the  first  space  vehicle. 
Delivery  of  the  prototype's  navigation  payload  was 
delayed  by  a  year,  which  the  program  office 
acknowledged  resulted  in  some  concurrency  in 
developing  and  integrating  the  prototype  and  the 
first  space  vehicle.  Program  officials  stated  that  the 
prototype  helped  the  program  identify  problems  and 
reduce  assembly  time  required  for  the  first  space 
vehicle. 

Other  Program  Issues 

The  program  office  recently  moved  its  estimate  for 
the  first  space  vehicle's  available  for  launch  date 
from  April  2014  to  April  2016 — a  24  month  delay. 
This  delay  was  in  part  driven  by  late  hardware 
deliveries  and  multiple  technical  problems  identified 
in  testing,  which  have  also  negatively  affected 
costs.  The  program  office  stated  that  the  navigation 
payload  for  the  first  GPS  satellite  was  delivered 
over  a  year  late.  The  program  is  still  working  to 
address  technical  problems  associated  with  the 


mission  data  unit,  the  brain  of  the  GPS  III  navigation 
mission.  Testing  revealed  radio  frequency  isolation 
issues,  which  can  result  in  signal  degradation. 

This  delay  more  closely  aligns  the  space  vehicle's 
delivery  with  the  availability  of  GPS  Ill's  ground 
system,  GPS  Operational  Control  Segment  (OCX). 
The  synchronization  of  GPS  Ill's  and  OCX's 
schedules  continues  to  be  a  risk.  GPS  III  satellites 
cannot  be  integrated  into  the  constellation  or  be 
considered  operational  until  OCX  Block  1  is 
delivered,  which  is  planned  for  October  2016. 

The  program  is  considering  dual  launching  GPS  III 
satellites,  potentially  beginning  as  early  as  the  fifth 
GPS  III  space  vehicle.  Our  prior  work  has  shown 
that  design  changes  during  production  can  have 
significant  cost  and  schedule  consequences, 
including  expensive  retrofitting  and  production 
delays. 

Program  Office  Comments 

The  Air  Force  also  provided  technical  comments, 
which  were  incorporated  where  deemed 
appropriate. 
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Ground/Air  Task  Oriented  Radar  (G/ATOR) 

The  Marine  Corps'  Ground/Air  Task  Oriented  Radar 
(G/ATOR)  is  an  active  electronic  scanned  array 
three-dimensional  short/medium  range  multi-role 
radar  designed  to  detect  cruise  missiles,  air 
breathing  targets,  rockets,  mortars,  and  artillery.  It 
replaces  five  different  legacy  radars.  G/ATOR  is  a 
block  acquisition;  later  blocks  are  mostly  software 
upgrades.  According  to  the  program  office, 
developmental  testing  concluded  in  August  2013, 
and  a  low-rate  production  decision  took  place  in 
January  2014,  followed  by  a  March  2014  contract 
award. 


Source:  U.S.  Marine  Corps. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Quantico,  VA 

08/2005 

09/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$372.4 

$981.1 

163.5 

R&D:  $285.9  million 

Procurement  cost 

$1,169.3 

$1,668.7 

42.7 

Procurement:  $2,089.4  million 

Total  program  cost 

$1,541.7 

$2,656.1 

72.3 

Total  funding:  $2,381.5  million 

Program  unit  cost 

$24,089 

$46,597 

93.4 

Procurement  quantity:  55 

Total  quantities 

64 

57 

-10.9 

Acquisition  cycle  time  (months) 

66 

133 

101.5 

The  G/ATOR  technologies  are  mature  and  its 
design  is  stable  and  demonstrated  but  its 
production  processes  are  not  yet  mature.  In 
addition,  the  program’s  reliability  is  beneath  the 
value  needed  to  meet  requirements  due  to 
software  issues.  To  address  this  shortfall,  the 
program  plans  to  develop  and  incorporate 
software  fixes  into  low-rate  initial  production. 
There  may  be  insufficient  time  to  test  all  fixes  prior 
to  the  start  of  production.  Production  processes 
were  demonstrated  on  a  pilot  production  line,  but 
they  did  not  demonstrate  stable,  adequate 
control.  The  program  office  intends  to  award  a 
contract  for  four  low-rate  production  systems  in 
March  2014.  According  to  the  program  office, 
these  systems  are  needed  to  conduct  operational 
and  environmental  testing  in  order  to  meet  the 
initial  operational  capability. 


Attainment  of  Product  Knowledge 


As  of  January  2014 

1  Resources  and  requirements  match 

•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

• 

•  Demonstrate  all  critical  technologies  in  a  realistic 
environment 

• 

•  Complete  preliminary  design  review 

• 

1  Product  design  is  stable 

•  Release  at  least  90  percent  of  design  drawings 

1 

• 

•  Test  a  system-level  integrated  prototype 

• 

1  Manufacturing  processes  are  mature 

•  Demonstrate  critical  processes  are  in  control 

o 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

• 

•  Test  a  production-representative  prototype 

• 

^  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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G/ATOR  Program 

Technology  Maturity 

All  six  G/ATOR  critical  technologies  are  fully  mature. 
However,  according  to  the  program  office,  testing 
revealed  issues  with  the  radar's  software  causing  it 
to  require  a  reboot  more  frequently  than  anticipated. 
The  problems  are  driven  by  a  number  of  lower-level 
software  issues  with  system  startup,  random 
crashes,  operator  control  console  freezes,  and  an 
unstable  command  and  control  interface.  The 
G/ATOR  program  office  has  put  together  a  plan  to 
incorporate  software  fixes  to  correct  system  start  up 
and  prevent  crashes.  Some  hardware  alterations 
may  be  required.  The  program  office  plans  to 
increase  and  improve  system  performance  by 
upgrading  the  software  integration  lab  to  support 
accelerated  testing  and  conducting  field  testing  with 
users  every  six  months  to  demonstrate  reliability 
growth  and  operational  relevance.  In  addition,  the 
gallium  nitride  technology,  which  will  be  used  to 
upgrade  the  radar's  transmit  and  receiving  modules, 
is  maturing  on  schedule.  This  new  semi-conductor 
technology  is  expected  to  achieve  better 
performance  at  a  lower  cost  and  will  be  installed  on 
the  last  low-rate  production  lot. 

Design  Maturity 

According  to  the  program  office,  the  G/ATOR  design 
is  mature,  with  100  percent  of  design  drawings 
released  and  testing  of  system-level  prototypes 
completed  in  August  2013,  well  after  the  critical 
design  review  but  before  the  program  enters 
production.  Risk  of  additional  design  changes 
exists,  due  to  modifications  planned  for  the 
production  units.  If  these  modifications  are  not 
incorporated  into  the  design  and  tested  before 
beginning  production,  there  may  be  schedule 
delays  if  the  program  chooses  to  continue 
production  and  retrofit  systems  at  a  later  date. 

Production  Maturity 

According  to  the  program  office,  the  G/ATOR 
production  readiness  review  was  completed  in 
September  2013  and  the  program  demonstrated 
that  its  manufacturing  processes  meet  DOD's 
requirement  for  the  start  of  low-rate  production. 
However,  the  manufacturing  readiness  level  does 
not  indicate  that  its  production  processes  are  stable 
and  in  control  as  preferred  by  commercial  best 
practices.  The  program  plans  to  begin  low-rate 
production  with  four  systems  in  March  2014. 


According  to  program  officials,  these  systems  are 
needed  to  conduct  developmental,  operational,  and 
environmental  qualification  testing  in  order  to  meet 
the  scheduled  Block  I  initial  operational  capability. 

Other  Program  Issues 

The  program  is  authorized  to  procure  57  G/ATOR 
systems;  however,  only  45  were  funded  in  the  fiscal 
year  2014  President's  budget.  According  to  the 
program  office,  the  12  unfunded  G/ATOR  systems 
will  require  funding  by  fiscal  year  201 6  in  order  to 
meet  initial  operational  capability.  In  addition,  the 
concurrent  development  and  production  of  G/ATOR 
may  be  adversely  affected  by  personnel  shortages 
caused,  in  part,  by  the  impending  retirement  of 
highly  experienced  acquisition  workforce  staff. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assesment,  the 
program  office  noted  that  in  October  2011 , 
G/ATOR's  performance  requirements  were  revised, 
which  resulted  in  a  re-designation  of  the  program  to 
a  major  defense  acquisition  program.  At  that  time, 
cost  estimates  were  rebaselined;  since  then 
G/ATOR  has  remained  on  schedule  and  its  total 
estimated  cost  decreased.  This  reduction  is  due,  in 
part,  from  implementing  better  buying  power  and 
should  cost  initiatives.  Technical  comments  were 
provided  by  the  program  office,  which  were 
incoporated  where  deemed  approrpiate. 
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Integrated  Air  and  Missile  Defense  (IAMD) 

The  Army's  Integrated  Air  and  Missile  Defense 
(IAMD)  program  is  being  developed  to  network 
sensors,  weapons,  and  a  common  battle  command 
system  across  an  integrated  fire  control  network  to 
support  the  engagement  of  air  and  missile  threats. 

The  IAMD  Battle  Command  System  (IBCS)  will 
provide  a  capability  to  control  and  manage  IAMD 
sensors  and  weapons,  such  as  the  Sentinel  radar 
and  Patriot  launcher  and  radar,  through  an  interface 
module  that  supplies  battle  management  data  and 
enables  networked  operations. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Space  &  Mission  Systems  Corp, 

12/2009 

12/2012 

change 

Raytheon  Integrated  Defense  Systems 

Research  and  development  cost 

$1,677.7 

$2,405.0 

43.4 

Program  office:  Huntsville,  AL 

Procurement  cost 

$3,611.0 

$3,399.0 

-5.9 

Funding  needed  to  complete: 

Total  program  cost 

$5,288.7 

$5,804.0 

9.7 

R&D:  $1,188.0  million 

Program  unit  cost 

$17,867 

$12,984 

-27.3 

Procurement:  $3,399.0  million 

Total  quantities 

296 

447 

51.0 

Total  funding:  $4,587.0  million 

Acquisition  cycle  time  (months) 

80 

81 

1.2 

Procurement  quantity:  431 

IAMD  completed  its  critical  design  review  in  May 
2012  with  a  stable  design  and  technologies 
nearing  full  maturity,  but  does  not  plan  to 
demonstrate  the  design  can  perform  as  expected 
until  February  2014.  lAMD's  mission  has  not 
changed,  but  changes  to  its  plans  for  integrating 
with  other  systems  has  significantly  increased  the 
size  of  its  software  effort,  delayed  its  subsystem 
design  reviews  and  increased  development  costs 
by  over  $717  million.  These  changes  include 
adding  Patriot  launcher  and  radar  functionality 
directly  onto  the  integrated  fire  control  network 
and  increasing  the  number  of  units  by  146  to  431 
total  units.  WIN-T  integration  is  also  a  significant 
risk. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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IAMD  Program 

Technology  Maturity 

Program  officials  estimate  that  IAMD  technologies 
will  not  be  fully  mature  until  its  planned  production 
decision  in  2015.  The  IAMD  program  entered 
system  development  in  December  2009  with  its  four 
critical  technologies — integrated  battle  command, 
integrated  defense  design,  integrated  fire  control 
network,  and  distributed  track  management — 
nearing  maturity,  according  to  an  Army  technology 
readiness  assessment  based  on  a  notional  design. 
The  Army  updated  the  technology  readiness 
assessment  in  March  2011  based  on  the  winning 
contractor's  design  and  reached  the  same 
conclusion  about  the  technologies'  maturity.  The 
Office  of  the  Assistant  Secretary  of  Defense  for 
Research  and  Engineering  concurred  with  the 
assessment,  but  noted  that  integration  with  the 
Warfighter  Information  Network-Tactical  (WIN-T)  is 
a  significant  risk.  It  also  noted  that  the  assessment 
was  based  on  modeling  and  simulations  of  WIN-T 
and  assumptions  about  performance.  As  a  result,  it 
recommended  realistic,  full-scale  testing  with  WIN-T 
prior  to  a  production  decision. 

Design  Maturity 

While  the  IAMD  program  has  released  nearly  all  of 
its  total  expected  drawings  and  held  a  system  level 
design  review  in  May  2012,  software  problems  and 
the  Army's  decision  to  operationalize  the  32nd  Test 
Battalion  could  cause  further  schedule  delays  and 
add  substantial  costs.  For  example,  as  a  result  of 
software  development  problems  associated  with  the 
integrated  battle  command  system,  the  IAMD 
program  realigned  software  deliveries  with  key  test 
activities.  The  revised  software  plan  was  approved 
in  April  2013,  resulting  in  a  program  delay  of  four 
months.  Failure  to  properly  develop,  test,  and 
integrate  software  could  result  in  delay  of 
capabilities  being  fielded  to  the  warfigther.  Also,  the 
operationalization  of  the  Patriot  Test  Battalion 
removes  over  50  percent  of  the  resources  for  both 
the  Patriot  and  IAMD  test  programs.  The  IAMD  and 
Patriot  programs  are  working  with  the  Army's  test 
group  to  consider  several  options  to  mitigate 
potential  risks,  including  delaying  all  test  activities 
until  Patriot  test  activities  are  complete  and 
conducting  tests  with  fewer  assets.  Further,  tests  of 
a  fully  integrated  system-level  prototype  are  not 
expected  to  begin  until  February  2014,  over  20 
months  after  design  review. 


Other  Program  Issues 

lAMD's  development  costs  have  risen  by  about  48 
percent,  or  over  $717  million,  since  beginning 
development  and  may  increase  further.  The  Army 
restructured  the  program  in  2011  to  incorporate  the 
Patriot  launchers  and  radars  directly  on  the 
integrated  fire  control  network  and  to  increase  the 
number  of  units  procured  from  285  to  431  to 
account  for  adjustments  to  the  battalion  force 
structure  and  accelerating  the  fielding  of 
capabilities.  However,  overall  procurement  costs 
have  decreased  by  $194  million.  According  to 
program  officials,  increasing  the  number  of  units  will 
provide  for  a  common  command  and  control  at  all 
organizational  levels.  Program  officials  estimate  the 
size  of  the  software  development  effort  at  over  6.6 
million  lines  of  code — a  37  percent  increase  over 
the  estimate  at  development  start.  In  addition,  about 
63  percent  of  this  code  will  be  newly  developed 
code  or  auto-generated  code  which  requires  less 
effort  to  develop  than  newly  developed  or  modified 
code. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  since  the  software  replan 
in  early  2013,  deliveries  of  software  have  been  on 
schedule  allowing  for  successful  demonstration  of 
the  IAMD  hardware/software  architecture  in  late 
2013  to  the  user.  This  demonstration  also  served  as 
risk  mitigation  for  early  integration  leading  up  to  the 
developmental  test  program  and  limited  user  test  in 
fiscal  year  2014  and  fiscal  year  2015.  Active  risk 
plans  are  in  place  to  manage  program  risks  and  the 
program  office  continues  to  work  with  other 
programs  which  IAMD  is  dependent  on  to  ensure 
technical  and  programmatic  risks  are  managed 
properly  across  programs.  The  loss  of  the  test 
battalion  will  potentially  result  in  some  changes  to 
the  IAMD  test  program,  but  those  changes  are 
actively  being  worked  with  all  stakeholders. 
Currently,  it  does  not  appear  this  resource  challenge 
will  affect  lAMD's  schedule.  All  program  changes 
depicted  in  this  report  are  captured  in  appropriate 
acquisition  decision  memorandums  and  the 
program's  acquisition  program  baseline.  The 
program  office  also  provided  technical  comments 
which  were  incorporated  where  deemed 
appropriate. 
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Joint  Air-to-Surface  Standoff  Missile  Extended  Range  (JASSM-ER) 


The  Air  Force's  JASSM-ER  program  plans  to  field  a 
next-generation  cruise  missile  capable  of  destroying 
the  enemy's  war-sustaining  capability  from  outside 
its  air  defenses.  JASSM-ER  missiles  are  low- 
observable,  subsonic,  and  have  a  range  of  greater 
than  500  miles.  They  provide  both  fighter  and 
bomber  crews  the  ability  to  strike  heavily  defended 
targets  early  in  a  campaign.  JASSM-ER  is  a  follow- 
on  program  to  the  JASSM  baseline  program.  The 
two  missiles'  hardware  is  70  percent  common  and 
their  software  is  95  percent  common. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Lockheed  Martin 

As  of 

Latest 

Percent 

Program  office:  Eglin  AFB,  FL 

04/2011 

08/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$286.8 

$253.6 

-11.6 

R&D:  $25.1  million 

Procurement  cost 

$3,636.3 

$3,670.3 

0.9 

Procurement:  $3,468.3  million 

Total  program  cost 

$3,923.1 

$3,923.9 

0.0 

Total  funding:  $3,493.4  million 

Program  unit  cost 

$1,550 

$1,318 

-15.0 

Procurement  quantity:  2,847 

Total  quantities 

2,531 

2,978 

17.7 

Acquisition  cycle  time  (months) 

NA 

130 

NA 

The  JASSM-ER  program  has  experienced 
production  issues  that  have  resulted  in  delays  to 
the  full-rate  production  decision  and  may  delay 
the  deliveries  required  to  provide  for  initial 
availability.  The  program  has  yet  to  accept  any 
ER  missile  deliveries  due  to  these  production 
issues.  Although  significant,  production  issues 
are  primarily  limited  to  the  new  ER  missile  engine, 
as  the  rest  of  baseline  and  ER  missile  production 
has  generally  proceeded  on  schedule. 
Sequestration  cuts  for  fiscal  year  2013  have 
resulted  in  the  Air  Force  deferring  some  missile 
purchases  beyond  fiscal  year  2017,  expanded 
obsolescence  risk,  and  caused  an  increase  in 
unfunded  requirements. 
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JASSM-ER  Program 

Technology  Maturity 

According  to  the  program  office,  JASSM-ER's  five 
critical  technologies — the  engine  lube  system, 
engine  system,  fuze,  low  observable  features,  and 
global  positioning  system — are  mature  and  have 
been  tested  in  a  realistic  environment  using  a 
production-representative  test  missile. 

Design  Maturity 

JASSM-ER's  design  is  currently  stable  and  has 
been  successfully  demonstrated  in  operational 
tests.  According  to  the  program  office,  the  number 
of  configuration  changes  has  decreased  overtime 
with  only  one  significant  design  change  in  the  last 
three  years. 

Production  Maturity 

The  full-rate  production  decision  for  JASSM-ER  has 
been  delayed  due  to  production  issues  which  may 
also  delay  initial  availability  for  the  assets  required. 
The  program  has  yet  to  accept  any  ER  missile 
deliveries  because  of  these  production  issues. 

The  most  significant  of  these  issues  is  with  the 
engine  lubrication  pump.  Program  officials  stated 
they  found  that  for  some  of  the  engine  pumps 
produced,  the  internal  metal  components  within  the 
pump  were  chafing,  releasing  metal  into  the 
lubrication  that  could  reduce  performance.  The 
program  plans  to  change  the  component  design  and 
implement  changes  during  production  ramp  up 
while  relying  on  the  current  design  and  production 
line  screening  to  remove  faulty  pumps  in  low-rate 
production.  Also,  the  program  experienced  a 
supplier  issue  with  the  fuel  control  unit — an  engine 
component  that  controls  the  fuel  throttle,  affecting 
missile  speed  and  acceleration.  They  discovered 
the  component  supplier  had  used  diluted  glue  which 
was  weaker  than  what  had  been  used  during 
component  qualification  testing.  Program  officials 
stated  all  30  of  the  first  ER  missiles  produced  are 
potentially  affected  and  they  placed  a  hold  on 
accepting  ER  missile  deliveries  until  an  agreement 
is  reached  with  Lockheed  Martin  on  how  to  address 
those  missiles. 

Aside  from  engine  lube  pump  and  fuel  control  unit 
issues,  the  program  states  production  of  the 
remaining  98  percent  of  ER  subsystems  and 
components  have  generally  proceeded  according  to 


schedule.  Production  of  the  JASSM  baseline  variant 
has  also  largely  been  unaffected  by  ER  production 
issues.  Production  of  Lot  10  baseline  missiles 
began  in  September  2013  with  missile  deliveries 
expected  to  begin  in  January  2014  while  start  of 
low-rate  production  of  the  second  lot  of  ER  missiles 
was  delayed  until  January  2014. 

Other  Program  Issues 

According  to  the  program  office,  sequestration  cuts 
for  fiscal  year  2013  have  resulted  in  the  Air  Force 
deferring  some  JASSM  purchases  to  the  out-years, 
increasing  obsolescence  risk,  and  increasing 
unfunded  requirements.  Prior  to  sequestration  cuts, 
the  program  planned  to  reach  its  economic  order 
quantity  of  360  missiles  per  year  in  fiscal  year  2016 
but  now  plans  to  purchase  missiles  at  the  minimum 
sustainment  rate  for  the  foreseeable  future.  The 
program  has  some  obsolescence  issues  that  are 
not  funded  because  of  sequestration  cuts, 
according  to  the  program,  which  could  lead  to 
production  gaps.  Also,  the  program  maintains 
sequestration  cuts  have  resulted  in  unfunded 
requirements,  such  as  warranties  for  upcoming 
missile  purchases,  production  tooling  for  full  rate 
production  ramp  up,  and  some  evaluation  testing. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the  Air 
Force  provided  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Joint  Light  Tactical  Vehicle  (JLTV) 

The  Army  and  Marine  Corps'  JLTV  is  a  family  of 
vehicles  being  developed  to  replace  the  High 
Mobility  Multipurpose  Wheeled  Vehicle  (HMMWV) 
for  some  missions.  The  JLTV  is  expected  to  provide 
better  protection  for  passengers  against  current  and 
future  battlefield  threats,  increased  payload 
capacity,  and  improved  automotive  performance 
over  the  up-armored  HMMWV;  it  must  also  be 
transportable.  Two-  and  four-  seat  variants  are 
planned  with  multiple  mission  configurations. 


Sources:  JPO  JLTV  EMD  Industry  Contractors  (AM  General,  Oshkosh  Corp.  &  Lockheed  Martin) 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  AM  General, 

As  of 

Latest 

Percent 

Lockheed  Martin,  Oshkosh 

10/2012 

07/2013 

change 

Program  office:  Harrison  Township,  Ml 

Research  and  development  cost 

$1,001.1 

$981.0 

-2.0 

Funding  needed  to  complete: 

Procurement  cost 

$22,661.3 

$22,651.7 

0.0 

R&D:  $377.3  million 

Total  program  cost 

$23,699.8 

$23,650.5 

-0.2 

Procurement:  $22,651.7  million 

Program  unit  cost 

$0,433 

$0,432 

-0.2 

Total  funding:  $23,046.9  million 

Total  quantities 

54,730 

54,730 

0.0 

Procurement  quantity:  54,599 

Acquisition  cycle  time  (months) 

125 

125 

0.0 

JLTV's  two  critical  technologies  have  been 
demonstrated  in  a  realistic  environment  on 
prototype  vehicles,  according  to  program  officials. 
In  lieu  of  a  critical  design  review,  the  program  held 
a  design  understanding  review  in  January  2013, 
and  according  to  program  documents,  confirmed 
that  all  three  contractors  had  more  than  90 
percent  of  design  files  under  configuration  control. 
A  recent  manufacturing  readiness  assessment 
concluded  that  all  three  vendors  were  on  track  to 
achieve  production  maturity  by  low-rate  initial 
production.  All  variants  are  envisioned  to  meet 
transportability  requirements,  and  testing  is 
ongoing  to  confirm  this  with  production 
representative  versions 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■■■•  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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JLTV  Program 

Technology  Maturity 

According  to  the  program  office,  its  critical 
technologies — underbelly  protection  armor  and 
side-kit  armor — are  fully  mature.  According  to 
program  officials,  ballistic  testing  conducted  since 
November  2012 — including  armor  samples,  vehicle 
cabs,  and  chassis — confirmed  this,  and  both 
technologies  have  been  demonstrated  in  fielded 
systems.  However,  integrating  the  technologies  to 
meet  all  system-level  requirements  for 
transportability,  protection,  and  mobility  could  prove 
challenging, 

Design  Maturity 

The  program  office  did  not  hold  a  formal  critical 
design  review  for  JLTV,  but  conducted  design 
understanding  reviews  with  contractors  between 
December  2012  and  January  2013.  According  to 
program  officials,  these  reviews  were  at  a  level  of 
detail  similar  to  a  critical  design  review  and  were 
held  to  support  the  requirement  for  mature  vehicle 
designs  at  the  time  of  contract  award  for  system 
development.  At  the  time  of  these  reviews,  all  long- 
lead  and  critical  designs  had  been  completed  and 
all  three  contractors  had  more  than  90  percent  of 
the  design  files  under  configuration  control.  The 
Office  of  the  Deputy  Assistant  Secretary  of  Defense 
(Systems  Engineering)  affirmed  that  the  exit  criteria 
for  system  development  had  been  satisfied. 
According  to  program  documentation,  contractors 
will  maintain  responsibility  for  configuration  control 
and  will  inform  the  government  of  changes.  The 
government  will  assume  control  of  the  design  after 
the  planned  award  of  one  low-rate  initial  production 
contract  in  2015. 

Production  Maturity 

The  Army  does  not  intend  to  use  process  capability 
index  data  to  assess  production  maturity,  as 
recommended  by  our  best  practices.  Instead,  it 
intends  to  use  other  metrics,  such  as  predicted 
assembly  times  and  feasibility  studies  to  assess 
production  maturity  by  the  production  readiness 
review,  scheduled  for  September  2014.  The 
program  recently  conducted  a  manufacturing 
readiness  assessment  with  all  three  vendors  to 
evaluate  manufacturing  readiness,  including  a  self- 
assessment  of  manufacturing  readiness  levels  by 
the  vendors.  The  program  office  believes  all  three 
vendors  are  on  track  to  be  production  capable  by 


low-rate  initial  production  contract  award  in  2015. 
Production  processes  are  not  expected  to  be  fully  in 
control  as  program  officials  noted  that  tooling  will 
not  be  fully  documented  until  this  point  due  to  the 
large  capital  investment  required. 

Other  Program  Issues 

On  the  basis  of  the  knowledge  gained  from  the 
technology  development  phase,  the  Army  and 
Marine  Corps  concluded  the  JLTV  could  not  meet 
requirements  for  both  protection  levels  and 
transportability  because  of  weight  and  relaxed  the 
requirement  to  transport  the  vehicle  by  helicopter  at 
high  altitude  and  at  certain  temperatures.  Program 
officials  acknowledge  some  risk  still  exists  in 
vehicles  achieving  relaxed  weight  requirements,  but 
the  prototype  weigh-ins  were  on  track,  thus 
diminishing  risk.  Prototype  vehicle  testing  will 
continue  through  October  2014,  and  the  results  will 
be  used  to  determine  whether  the  vehicles  meet 
system  requirements.  Army  officials  recognize  that 
the  test  schedule  is  compressed,  and  that  slips  in 
the  limited  user  test  could  result  in  a  slip  of  the 
program's  production  decision  planned  for  July 
2015.  However,  program  officials  believe  that  the 
results  of  testing  will  be  available  in  sufficient  time. 

At  the  end  of  the  system  development  phase,  a  low- 
rate  initial  production  contract  will  be  awarded  to  a 
single  source.  The  program  office  will  seek  approval 
to  limit  competition  to  the  three  current  vendors,  but 
if  not  approved  will  revert  to  a  full  and  open 
competition  strategy.  The  program  office  does  not 
believe  there  are  any  viable  sources  for  JLTV 
production  other  than  the  three  current  vendors. 

Finally,  the  Army  plans  to  begin  procuring  JLTV 
while  simultaneously  procuring  the  Armored  Multi- 
Purpose  Vehicle.  The  simultaneous  procurement  of 
these  programs  is  expected  to  continue  for  a 
decade  or  more. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS  Inc  1A) 

JPALS  Increment  1  is  a  Navy-led  program  to 
develop  a  GPS-based  aircraft  landing  system  to 
replace  current  radar-based  systems  on  its  ships.  It 
is  designed  to  provide  reliable  precision  approach 
and  landing  capability  in  adverse  environmental 
conditions  and  improved  interoperability.  Increment 
1A  is  the  ship-based  system  and  increment  IB  will 
integrate  JPALS  with  sea-based  aircraft.  Both  are 
needed  to  provide  the  full  capability.  We  assessed 
increment  1A  and  made  comments  on  increments 
IB,  2,  and  other  follow-on  efforts. 


GPS  Satellites 


Source:  U.S.  Navy. 


System  development 


> 


Production 


▲ 

▲ 

▲ 

A 

A  A 

A 

Development 

Design 

GAO 

Low-rate 

Start  Initial 

Full-rate 

start 

review 

review 

decision 

operational  test  capability 

decision 

(7/08) 

(12/10) 

(1/14) 

(TBD) 

(1/16)  (7/16) 

(8/16) 

► 


Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Lexington  Park,  MD 

07/2008 

09/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$833.1 

$859.3 

3.1 

R&D:  $115.8  million 

Procurement  cost 

$224.3 

$257.7 

14.9 

Procurement:  $257.7  million 

Total  program  cost 

$1,064.7 

$1,124.2 

5.6 

Total  funding:  $373.5  million 

Program  unit  cost 

$28,775 

$30,385 

5.6 

Procurement  quantity:  27 

Total  quantities 

37 

37 

0.0 

Acquisition  cycle  time  (months) 

75 

96 

28.0 

JPALS  Increment  lA's  two  critical  technologies 
are  mature  and  have  been  demonstrated  in  a 
realistic  environment.  Program  officials  reported 
completing  baseline  software  development  as  of 
April  2012.  The  program  began  system-level 
development  testing  in  July  2012  and  sea-based 
testing  in  December  2012.  The  program 
completed  108  integrated  flight  tests  with  no 
major  anomalies  reported.  Increment  lA's 
production  decision  scheduled  for  November 
2013  was  cancelled  due  to  budget  reductions, 
with  a  new  date  to  be  determined.  Ship  availability 
delays  for  installation  and  procurement  changes 
resulted  in  schedule  and  cost  breaches, 
respectively,  of  thresholds  in  the  program's 
baseline.  Increment  IB  is  scheduled  to  start 
system  development  in  2015.  Lead  responsibility 
for  Increment  2,  land-based  JPALS,  transferred 
from  the  Air  Force  to  the  Navy  beginning  in  fiscal 
year  2014. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

Q  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  JPALS  Inc  1A 


JPALS  Inc  1A  Program 

Technology  Maturity 

JPALS  Increment  lA's  two  critical  technologies  are 
mature  and  have  been  demonstrated  in  a  realistic 
environment.  Program  officials  reported 
demonstrating  the  technologies  during  sea-based 
flight  testing  in  2013.  JPALS  functionality  is  primarily 
software-based,  and  the  program  reported  that  its 
software  development  and  integration  effort  was 
complete  as  of  April  2012.  The  program  will 
continue  developmental  testing,  with  operational 
testing  planned  to  begin  in  2016. 

Design  Maturity 

The  JPALS  program  held  its  critical  design  review  in 
December  2010  and  released  all  of  its  expected 
design  drawings  to  manufacturing  at  that  time.  The 
JPALS  design  is  currently  stable,  and  the  program 
began  system-level  development  testing  of  a 
prototype  in  July  2012  to  show  that  it  will  perform  as 
expected.  Sea-based  testing  began  in  December 
2012  and  program  officials  reported  completing  108 
integrated  flight  tests  as  of  July  201 3,  with  no  major 
anomalies  identified.  Shore-based  auto-landing 
demonstrations  began  in  October  2013,  and 
program  officials  reported  completing  70  ship-based 
auto-landing  demonstrations  as  of  November  2013. 

Production  Maturity 

According  to  JPALS  program  officials,  the  program 
has  not  identified  any  critical  manufacturing 
processes  and  the  system's  hardware  is  comprised 
primarily  of  off-the-shelf  components.  The  program 
has  accepted  delivery  of  all  eight  of  its  engineering 
development  models,  seven  of  which  are  production 
representative. 

Other  Program  Issues 

The  program's  production  decision,  scheduled  for 
November  2013,  was  cancelled  and  a  new  date  has 
yet  to  be  determined.  Program  officials  stated  that 
budget  reductions  due  to  sequestration  eliminated 
funding  for  the  low-rate  initial  production  contract. 
They  noted  that  a  new  date  will  not  be  scheduled 
until  the  Navy  resolves  this  funding  issue.  The 
program  previously  delayed  the  production  decision 
from  May  2013  to  November  2013  because  the 
aircraft  carrier  needed  to  conduct  ship-based 
integrated  testing  was  not  available  for  installation. 
As  a  result,  the  program  reported  a  schedule  breach 
of  the  acquisition  program  baseline. The  program 


also  extended  planned  production  to  avoid 
acquiring  systems  earlier  than  needed  and  moved 
the  purchase  of  one  unit  from  development  to 
procurement.  These  changes  increased 
procurement  costs  and  resulted  in  a  cost  breach  of 
a  threshold  set  in  the  acquisition  baseline. 

System  development  for  JPALS  Increment  1 B — 
which  will  integrate  JPALS  with  sea-based  aircraft — 
is  scheduled  for  2015.  Program  officials  stated  that 
increment  1 B  will  be  developed  to  provide  JPALS 
functionality  in  a  packaged  way  that  allows  for 
integration  with  any  aircraft  rather  than  tailoring 
efforts  to  each  individual  platform.  Increment  2  is  the 
land-based  version  of  JPALS  and  system 
development  is  scheduled  for  2015.  Beginning  with 
fiscal  year  2014,  lead  service  responsibility  for 
JPALS  Increment  2  transferred  from  the  Air  Force  to 
the  Navy. 

Program  officials  stated  that  the  recently  conducted 
successful  auto-land  demonstrations  may  allow  for 
the  incorporation  of  capabilities  planned  for  future 
increments.  Incorporating  auto-land  and  unmanned 
aerial  vehicle  capabilities  into  increments  1 A  and  1 B 
will  provide  enabling  technology  for  the  Navy's 
future  ship  classes  and  the  Unmanned  Carrier- 
Launched  Surveillance  and  Strike  program  for 
which  JPALS  was  selected  as  the  primary  landing 
system. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
JPALS  program  office  noted  that  it  concurred  with 
our  review  and  that  flight  testing  was  highly 
successful  and  the  system  performed  as  intended 
with  no  significant  design  or  performance  issues 
identified  to  date.  The  program  is  working 
aggressively  with  the  Navy  and  the  Office  of 
Secretary  of  Defense  to  resolve  the  cost  and 
schedule  breaches,  and  anticipates  corrective 
actions  will  be  incorporated  in  the  fiscal  year  2015 
President's  budget  submission.  The  progam  office 
also  provided  technical  comments,  which  were 
incorporated  where  deemed  appropriate. 
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Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios  (JTRS  HMS) 

DOD's  JTRS  program  is  developing  software- 
defined  radios  that  will  interoperate  with  existing 
radios  and  increase  communications  and 
networking  capabilities.  The  JTRS  HMS  program  is 
currently  developing  two  radios:  the  Rifleman  radio 
for  unclassified  use  and  the  Manpack  radio  for  use 
in  a  classified  domain.  A  subset  of  the  Manpack 
radios  will  be  interoperable  with  the  Mobile  User 
Objective  System  (MUOS),  a  satellite 
communication  system. 


Source:  ©  2012  General  Dynamics  C4S. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  General  Dynamics  C4 

As  of 

Latest 

Percent 

Systems,  Inc. 

05/2004 

09/2013 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$572.9 

$1,255.8 

119.2 

Ground,  MD 

Procurement  cost 

$9,984.0 

$7,680.2 

-23.1 

Funding  needed  to  complete: 

Total  program  cost 

$10,556.9 

$8,935.9 

-15.4 

R&D:  $31.3  million 

Program  unit  cost 

$0,032 

$0,033 

2.6 

Procurement:  $6,706.3  million 

Total  quantities 

328,674 

271,202 

-17.5 

Total  funding:  $6,737.7  million 

Acquisition  cycle  time  (months) 

85 

112 

31.8 

Procurement  quantity:  236,240 

The  JTRS  HMS  program  continues  to  conduct 
operational  testing  on  both  the  Rifleman  and 
Manpack  variants  and  has  demonstrated  full 
maturity  of  all  technologies,  but  has  not 
demonstrated  its  production  processes.  Program 
officials  have  taken  steps  to  address  the  Manpack 
reliability  shortfalls  identified  during  operational 
testing  in  May  2012  and  plan  to  demonstrate  the 
radio's  capabilities  during  follow-on  testing  in  May 
2014.  Full-rate  production  decisions  for  both 
variants  have  been  delayed  to  fiscal  year  2015 
because  of  a  change  in  the  program's  acquisition 
strategy  to  allow  new  vendors  to  compete  for  the 
contracts.  In  December  2013,  the  program 
received  approval  to  increase  the  low-rate  initial 
production  quantity  for  the  Manpack  radio.  The 
program  is  working  with  the  Army  to  manage 
capability  gaps  resulting  from  both  radios'  full-rate 
production  delays. 
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Common  Name:  JTRS  HMS 


JTRS  HMS  Program 

Technology  Maturity 

According  to  a  program  official,  JTRS  HMS  has  fully 
demonstrated  the  maturity  of  all  its  critical 
technologies  for  both  the  Rifleman  and  Manpack 
radios.  The  program  has  conducted  operational 
testing  on  both  variants  for  over  four  years,  and 
plans  to  conduct  additional  testing  in  fiscal  year 
2014. 

Design  Maturity 

According  to  program  officials,  the  designs  of  both 
variants  are  stable,  but  reliability  issues  for  the 
Manpack  radio  could  require  design  modifications. 
Following  operational  testing  in  May  2012,  DOD  test 
officials  reported  that  the  Manpack  was  not 
operationally  effective  or  suitable  because  it  failed 
to  demonstrate  a  reliability  requirement.  Program 
officials  said  they  have  taken  steps  to  address  the 
Manpack's  reliability  shortfalls  and  plan  to 
demonstrate  all  remaining  capabilities  during  follow- 
on  testing  in  May  2014. 

Production  Maturity 

According  to  JTRS  HMS  officials,  the  program's 
manufacturing  readiness  is  at  a  level  consistent  with 
DOD  guidance,  but  not  yet  at  a  level  that 
demonstrates  that  its  critical  manufacturing 
processes  are  in  control. 

Other  Program  Issues 

Both  variants  of  JTRS  HMS  have  experienced 
additional  schedule  slips  since  last  year's 
assessment.  The  program  revised  its  acquisition 
strategy  following  an  Office  of  the  Secretary  of 
Defense  directive  that  JTRS  HMS  conduct  full  and 
open  competition  for  the  full-rate  contracts  for  both 
variants.  As  a  result,  the  program  office  expects  the 
full-rate  production  decisions  for  the  Rifleman  and 
Manpack  radios  to  slip  from  the  second  half  of  fiscal 
year  201 3  to  March  and  September  2015, 
respectively.  Prior  to  the  full-rate  production 
decisions,  the  program  must  solicit  proposals  from 
new  vendors  and,  according  to  program  officials, 
will  likely  award  new  contracts  and  test  new  radios. 
The  Army  would  have  to  demonstrate  that  these 
new  radios  meet  the  Rifleman  and  Manpack 
requirements  in  an  operationally-realistic 
environment.  This  delay  puts  the  program  nearly 
three  years  behind  the  originally  scheduled  full-rate 
production  decisions  for  the  Rifleman  radio  in  May 


2012  and  the  Manpack  radio  in  December  2012. 
The  program  manager  said  he  is  working  with  Army 
operations  and  resource  management  staff  to 
manage  capability  gaps  resulting  from  both  radios' 
full-rate  production  delays.  They  will  establish 
deployment  quantities  and  locations  for  the  HMS 
radios  procured  through  low-rate  initial  production  to 
meet  the  Army's  immediate  needs.  The  program 
received  approval  for  an  additional  lot  of  1,500 
Manpack  radios  in  December  2013,  but  has  not 
received  authorization  to  increase  the  low-rate  initial 
production  quantity  for  the  Rifleman  radio  over  the 
current  level  set  by  the  Office  of  the  Secretary  of 
Defense  in  July  2012. 

Additionally,  demonstrating  the  Manpack  with  the 
MUOS  antenna  and  waveform  remains  a  technical 
risk  for  the  program.  The  JTRS  HMS  program  is 
currently  scheduled  to  field  the  MUOS-capable 
Manpack  radios  in  2014,  but  officials  stated  that 
they  will  wait  to  demonstrate  the  Manpack  with  the 
MUOS  antenna  and  waveform  until  after  the  MUOS 
program  successfully  completes  its  own  operational 
testing,  which  is  scheduled  for  June  2014. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment, 
program  officials  noted  that  in  2012,  HMS  early 
operational  tests  were  conducted  on  low-rate  initial 
production  variants  of  the  HMS  Manpack  radio. 
Since  this  test,  numerous  corrected  deficiencies 
have  been  implemented,  including  production 
improvements  and  extended  environmental  stress 
screening,  significant  improvement  has  been 
documented  in  follow-on  performance  verification 
testing  and  will  be  formally  demonstrated  at  the 
Network  Integration  Evaluation  14.2  in  May  2014. 
HMS  Rifleman  and  Manpack  radios  have  been 
successfully  fielded  and  are  in  use  by  operational 
units  in  support  of  global  operations.  To  meet 
operational  needs,  in  December  2013,  the  Office  of 
the  Secretary  of  Defense  approved  an  additional 
low-rate  initial  production  allotment  of  HMS 
Manpack  radios,  while  also  indicating  support  for  a 
multi-vendor,  multi-award  construct  for  HMS  full  rate 
production. 
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KC-46  Tanker  Modernization  Program  (KC-46) 


The  Air  Force's  KC-46  program  plans  to  convert  an 
aircraft  designed  for  commercial  use  into  an  aerial 
refueling  tanker  in  order  to  refuel  Air  Force,  Navy, 
Marine  Corps,  and  allied  aircraft.  The  program  is  the 
first  of  three  planned  phases  to  replace  the  Air 
Force's  aging  fleet  of  KC-135  aerial  refueling 
tankers.  The  KC-46  has  been  designed  to  improve 
on  the  KC-135's  refueling  capacity,  efficiency, 
capabilities  for  cargo  and  aeromedical  evacuation, 
and  to  integrate  defensive  systems. 


Source:  ©2011  Boeing.  All  rights  reserved. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Boeing 

As  of 

Latest 

Percent 

Program  office:  Wright-Patterson  AFB, 

02/2011 

12/2012 

change 

OH 

Research  and  development  cost 

$7,206.3 

$7,045.5 

-2.2 

Funding  needed  to  complete: 

Procurement  cost 

$34,992.9 

$33,848.5 

-3.3 

R&D:  $3,355.6  million 

Total  program  cost 

$46,090.0 

$44,519.2 

-3.4 

Procurement:  $33,848.5  million 

Program  unit  cost 

$257,486 

$248,710 

-3.4 

Total  funding:  $40,829.3  million 

Total  quantities 

179 

179 

0.0 

Procurement  quantity:  175 

Acquisition  cycle  time  (months) 

78 

78 

0.0 

The  program  held  its  critical  design  review  in  July 
2013  with  its  critical  technologies  nearing  full 
maturity  and  its  design  stable  but  not 
demonstrated.  To  reduce  risk  and  improve 
schedule,  Boeing  decided  to  develop  situational 
awareness  software  internally  prior  to  the  critical 
design  review.  This  resulted  in  replacing  one  of 
the  program's  critical  technologies  and  re-using 
software  code.  Boeing  is  currently  manufacturing 
the  four  development  aircraft,  but  test  boom 
production  has  been  delayed  by  almost  a  year 
due  to  design  changes  and  late  parts.  The 
program  has  demonstrated  that  it  can  produce 
the  required  military  subsystems  in  a  production 
representative  environment,  but  these  processes 
will  not  be  in  control  by  the  low-rate  production 
decision.  The  Air  Force  and  Boeing  are  working 
through  software  and  hardware  integration  and 
flight  test  risks. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  KC-46 


KC-46  Program 

Technology  Maturity 

The  KC-46  entered  system  development  with  its 
critical  technologies — a  three-dimensional  display  to 
monitor  and  enable  aerial  refueling  activities  and  a 
software  module  designed  to  increase  situational 
awareness — nearing  full  maturity,  but  not  yet 
demonstrated  in  a  realistic  environment.  In  March 
2013,  almost  four  months  before  the  program's 
critical  design  review,  Boeing  decided  to  develop 
the  situational  awareness  software  internally. 
Boeing's  version  of  this  development  effort  replaced 
the  prior  software  modules  with  a  new  one.  The  new 
software  module  is  nearing  full  maturity,  but  has  not 
yet  been  demonstrated  in  a  realistic  environment. 
To  reduce  risk  and  improve  schedule,  Boeing  re¬ 
used  some  of  the  situational  awareness  software 
code  from  prior  programs.  The  program  plans  to 
fully  mature  its  critical  technologies  prior  to  its  low- 
rate  production  decision. 

Design  Maturity 

The  program  held  its  critical  design  review  in  July 
2013  with  over  90  percent  of  its  design  drawings 
releasable,  but  before  fully  maturing  its  technologies 
and  testing  an  aircraft  that  integrates  the  military 
sub-systems.  The  program  will  not  begin  testing  an 
aircraft  that  integrates  military  sub-systems  until 
January  2015,  18  months  after  its  critical  design 
review.  Best  practices  call  for  completing  a  full 
system-level  prototype  demonstration  by  the  critical 
design  review  in  order  to  show  that  the  design  is 
capable  of  meeting  performance  requirements. 

Production  Maturity 

Boeing  has  begun  production  of  the  four 
development  aircraft,  and  is  also  manufacturing  a 
test  refueling  boom  and  a  boom  for  the  first 
development  aircraft.  Test  refueling  boom 
production  has  been  delayed  by  almost  a  year  due 
to  design  changes  and  late  parts  deliveries.  Boeing 
officials  said  the  test  refueling  boom  is  a  risk 
reduction  effort  and  lessons  learned  will  be  applied 
to  future  boom  builds.  The  program  reported  that  it 
is  primarily  using  manufacturing  readiness  levels  to 
assess  manufacturing  risk.  As  of  July  201 3,  the 
program  demonstrated  that  it  is  capable  of 
producing  the  applicable  military  subsystems,  such 
as  several  aerial  refueling  and  defensive 
subsystems,  in  a  production  representative 
environment.  The  low-rate  initial  production  decision 


is  currently  scheduled  for  August  201 5,  but  the 
program  will  not  have  its  production  processes  for 
military  subsystems  in  control  at  that  time  as 
recommended  by  best  practices. 

Other  Program  Issues 

The  Air  Force  has  identified  software  and  hardware 
integration  and  the  aggressive  nature  of  the  KC-46 
flight  test  schedule  as  top  program  risks.  While  most 
software  and  hardware  integration  has  progressed 
according  to  plan,  these  efforts  have  largely 
addressed  commercial  rather  than  military  unique 
components  to  date,  and  some  aerial  refueling  and 
avionics  software  has  not  been  delivered  on  time. 
The  flight  test  pace  is  more  aggressive  than  other 
military  aircraft  development  programs,  and 
Boeing's  test  approach  requires  a  significant 
amount  of  coordination  among  key  stakeholders. 
The  program  office  and  Boeing  are  working  together 
to  mitigate  these  risks  by,  for  example,  conducting 
dry  runs  prior  to  flight  testing. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  stated  they  mitigated  risk  by  negotiating 
the  competitive  fixed-price  incentive  development 
contract  with  firm-fixed  and  not-to  exceed  pricing  for 
production.  The  program  noted  that  about  50 
percent  of  the  development  work  has  been 
completed  thus  far  and  Boeing  has  met  or 
exceeded  all  contractual  requirements.  The 
program  is  closely  tracking  software  as  a  risk, 
however,  it  does  not  believe  there  are  significant 
software-related  issues  at  this  time.  The  program 
further  stated  that  it  plans  to  mitigate  the  risk  posed 
by  concurrency  by  ensuring  that  adequate  testing  is 
completed  prior  to  the  production  decision.  In 
addition,  Boeing's  contract  requires  it  to  incorporate 
fixes  into  production  aircraft  at  no  additional  cost. 
DOD  and  the  Air  Force  also  provided  technical 
comments,  which  were  incorporated  as  appropriate. 
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LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

The  Navy's  LHA  6  class  will  replace  the  LHA  1 
Tarawa  class  amphibious  assault  ships.  LHA  6  class 
is  based  on  the  fielded  LHD  8  and  consists  of  three 
ships.  The  ships  will  feature  enhanced  aviation 
capabilities  and  are  designed  to  support  Marine 
Corps  assets  in  an  expeditionary  strike  group.  LHA 
6  construction  began  in  December  2008  and  ship 
delivery  is  expected  in  March  2014.  LHA  7 
construction  began  in  July  2013  and  delivery  is 
expected  in  December  2018.  The  Navy  intends  to 
award  the  construction  contract  for  LHA  8  in  fiscal 
year  2017. 


Source:  Huntington  Ingalls  Industries 


Concept  System  development  Production 


▲ 

▲ 

▲ 

▲ 

▲  ▲ 

▲ 

Program 

Contract 

Fabrication 

GAO 

Ship  Start 

Initial 

start 

award 

start 

review 

delivery  operational  test 

capability 

(7/01) 

(6/07) 

(1/08) 

(1/14) 

(3/14)  (6/14) 

(4/16) 

Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Huntington  Ingalls 

As  of 

Latest 

Percent 

Industries 

01/2006 

12/2012 

change 

Program  office:  Washington,  DC 

Research  and  development  cost 

$232.4 

$397.1 

70.9 

Funding  needed  to  complete: 

Procurement  cost 

$3,112.3 

$9,710.6 

212.0 

R&D:  $75.6  million 

Total  program  cost 

$3,344.6 

$10,109.7 

202.3 

Procurement:  $3,570.5  million 

Program  unit  cost 

$3,344,647 

$3,369,910 

0.8 

Total  funding:  $3,647.2  million 

Total  quantities 

1 

3 

200.0 

Procurement  quantity:  1 

Acquisition  cycle  time  (months) 

146 

177 

21.2 

LHA  6  began  construction  in  December  2008  with 
mature  technologies  and  a  design  that  was  only 
65  percent  complete.  After  a  19  month  delay,  the 
Navy  expects  delivery  will  occur  in  March  2014. 
LHA  6  will  also  incur  an  estimated  $42.4  million  in 
cost  growth  due  to  postdelivery  rework  of  the 
ship's  deck  to  cope  with  Joint  Strike  Fighter 
exhaust  and  downwash.  Construction  of  LHA  7 
(which  largely  shares  the  LHA  6  design)  began  in 
July  2013  and  the  program  office  believes  that  the 
issues  that  led  to  cost  increases  and  schedule 
delays  on  LHA  6  have  been  largely  resolved  and 
are  unlikely  to  cause  problems  for  LHA  7.  Design 
changes  to  LHA  8  are  more  significant  and 
include  the  addition  of  a  well  deck.  The  Navy 
plans  to  competitively  award  a  construction 
contract  for  LHA  8  and  has  awarded  contracts 
with  two  shipyards  to  assist  with  early  design 
work. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  LHA  6 


LHA  6  Program 

Technology,  Design,  and  Production  Maturity 

All  LHA  critical  technologies  were  mature  when  the 
program  awarded  its  construction  contract  in  June 
2007.  Although  not  considered  critical  technologies, 
the  program  has  identified  an  additional  six  key 
subsystems  necessary  to  achieve  capabilities.  Five 
of  these  subsystems  are  mature.  The  sixth,  the 
Joint  Precision  Approach  and  Landing  System,  is 
still  in  development,  but  LHA  6  can  use  backup 
aviation  control  systems  to  meet  requirements. 
There  are  no  new  critical  technologies  expected  for 
LHA  7  or  LHA  8,  but  requirements  for  LHA  8  are  still 
in  development. 

In  January  2014,  the  Navy  and  the  shipbuilder 
conducted  acceptance  trials  on  LHA  6  and  Navy 
officials  anticipate  ship  delivery  will  occur  in  March 
2014,  19  months  later  than  the  contracted  delivery 
date.  At  the  time  of  acceptance  trials,  the  LHA  6 
design  was  at  least  99  percent  complete  and 
construction  was  over  97  percent  complete.  LHA  6 
began  construction  in  December  2008  with  only  65 
percent  of  its  design  complete,  and  subsequent 
design  quality  issues  have  caused  more  design 
changes  than  anticipated  and  higher  levels  of 
rework.  Following  the  delivery  of  LHA  6,  the  Navy 
will  spend  an  estimated  $42.4  million  to  reconfigure 
the  ship's  flight  deck  to  cope  with  exhaust  and 
downwash  from  the  Joint  Strike  Fighter. 

Construction  of  LHA  7  (which  largely  shares  the 
same  design  as  LHA  6)  began  in  July  2013  and 
flight  deck  modifications  due  to  JSF  are  still  being 
incorporated  into  the  design.  Officials,  however,  do 
not  believe  that  this  will  interfere  with  the  ship's 
construction.  Other  design  changes  to  LHA  7 
include  a  new  firefighting  system  and  updates  to  the 
radar  and  the  command,  control,  communications, 
computers,  and  intelligence  systems.  Design 
changes  to  LHA  8  will  be  more  significant  as  the 
Navy  will  incorporate  a  well  deck  that  can 
accommodate  two  landing  craft.  Program  officials 
note  that  the  new  design  also  allows  for  better 
maintenance  of  aviation  capabilities.  The  program 
may  also  incorporate  a  flexible  infrastructure — 
which  would  allow  compartments  to  be  reconfigured 
to  meet  changing  mission  needs  with  less  rework — 
into  some  low  risk  areas  of  the  LHA  8  design  should 
the  Navy  determine  that  it  results  in  lifecycle  cost 
savings. 


Other  Program  Issues 

Program  officials  believe  that  as  a  result  of 
increasing  the  number  of  design  drawing  reviews 
and  ensuring  additional  lead  time  for  material  orders 
on  LHA  7,  the  program  will  not  experience  the  high 
level  of  rework  that  led  to  cost  growth  and  schedule 
delays  during  construction  of  LHA  6.  In  order  to 
improve  quality  and  shipyard  performance  on  LHA 
7,  the  Navy  included  contract  incentives  of  up  to  $41 
million;  although  officials  note  that  it  is  too  soon  to 
determine  whether  the  incentives  are  effective  at 
keeping  rework  rates  low. 

The  Navy  intends  to  competitively  award  a 
construction  contract  for  LHA  8,  and  in  November 
2012  awarded  early  industry  involvement  contracts 
to  two  shipbuilding  contractors  that  it  determined 
are  capable  of  building  the  ship  without  major 
recapitalization.  The  contracts  allow  the 
shipbuilders  to  participate  in  early  stage  design 
work  and  investigate  opportunities  for  potential  cost 
reductions,  and  were  recently  modified  to  extend 
the  period  of  performance  past  the  past  the  original 
April  2014  contract  expiration  date. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  program  office  stated  that  based  on  the 
success  of  LHA  6's  acceptance  trials,  the  Navy's 
independent  assessor — the  Board  of  Inspection  and 
Survey — recommended  that  the  Navy  accept 
delivery  of  the  ship.  The  extension  of  the  periods  of 
performance  of  the  early  industry  involvement 
contracts  for  LHA  8  efforts  will  allow  the  joint 
industry/government  team  to  continue  working  on 
identifying  innovative  ways  to  reduce  the  projected 
cost  of  the  ship.  The  program  office  also  provided 
technical  comments,  which  were  incorporated  as 
appropriate. 
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Littoral  Combat  Ship  (LCS) 

The  Navy's  LCS  is  designed  to  perform  mine 
countermeasures,  antisubmarine  warfare,  and 
surface  warfare  missions.  It  consists  of  the  ship 
itself,  or  seaframe,  and  the  mission  packages  it 
deploys.  The  Navy  bought  the  first  four  seaframes 
in  two  unique  designs — one  based  on  a  steel  semi- 
planing  monohull  (Freedom  variant)  and  the  other 
based  on  an  aluminum  trimaran  hull  (Independence 
variant) — and  subsequently  awarded  a  contract  for 
a  block  buy  of  up  to  10  ships  to  both  contractors. 

We  assessed  both  seaframe  designs 

Sources:  Lockheed  Martin  (left);  General  Dynamics  (right). 


Concept  System  development  Production 
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Program  Essentials 

Prime  contractor:  Austal  USA, 
Lockheed  Martin 

Program  office:  Washington  Navy  Yard, 
DC 

Funding  needed  to  complete: 

R&D:  $704.1  million 
Procurement:  $18,775.1  million 
Total  funding:  $19,631.6  million 
Procurement  quantity:  36 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

05/2004 

12/2012 

change 

$932.9 

$3,596.3 

285.5 

$496.0 

$26,214.6 

5,185.2 

$1,428.9 

$30,027.0 

2,001.4 

$357,225 

$577,442 

61.6 

4 

52 

1,200.0 

41 

122 

197.6 

Cost  data  are  for  the  seaframe  only.  Research  and  development  funding  includes  detail  design  and 
construction  of  two  ships.  DOD  recently  announced  plans  to  reduce  total  seaframe  quantities  to  32. 


The  LCS  seaframe  program  has  demonstrated 
the  maturity  of  16  of  its  18  critical  technologies 
and  continues  to  make  design  and  production 
process  changes.  Additional  design  changes 
have  been  endorsed  by  the  LCS  Council  for  fiscal 
year  2015:  bridge  wings  and  a  seven-meter  rigid 
hull  inflatable  boat  for  the  Independence  variant 
and  stronger  stern  ramp  for  the  Freedom  variant. 
The  Navy  accepted  delivery  of  LCS  4  in 
September  2013,  a  delay  of  6  months,  and  is  in 
steady  production  with  12  hulls  under  contract 
and/or  construction.  The  next  block-buy  is 
scheduled  for  fiscal  year  2016,  when  24 
seaframes  will  already  be  delivered,  constructed, 
or  under  contract. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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LCS  Program 

Technology  Maturity 

Sixteen  of  the  18  critical  technologies  for  both  LCS 
designs  are  mature  and  have  been  demonstrated  in 
a  realistic  environment.  The  two  remaining 
technologies — LCS  1's  overhead  launch  and 
retrieval  system  and  LCS  2's  aluminum  structure 
are  nearing  maturity — according  to  our  best  practice 
standards.  Though  program  officials  believe  that 
LCS  2's  aluminum  structure  hull  is  mature  as  the 
ship  is  operational,  there  are  still  unknowns  related 
to  the  hull  structure.  As  a  result,  full  maturity  will  not 
be  demonstrated  until  the  completion  of  shock  and 
survivability  trials  to  validate  survivability  and  the 
ship's  ability  to  achieve  a  20-year  service  life.  These 
tests  are  not  expected  to  begin  until  August  2015. 

Design  and  Production  Maturity 

The  Navy  started  construction  of  LCS  1  and  2 
without  a  stable  design  and  has  had  to  incorporate 
design  changes  on  follow-on  seaframes.  LCS  1  and 
LCS  2  are  still  undergoing  testing,  and  the  Navy  is 
incorporating  design  fixes  for  identified  deficiencies 
into  the  designs  of  follow-on  ships.  In  addition,  a 
series  of  additional  design  changes  for  both  variants 
have  been  approved  by  the  LCS  Council  for  fiscal 
year  2015;  including  bridge  wings  and  a  seven- 
meter  rigid  hull  inflatable  boat  for  the  Independence 
variant  and  stronger  stern  ramp  for  the  Freedom 
variant.  LCS  4  experienced  delays  and  was 
delivered  six  months  after  its  expected  contract 
delivery  date  of  March  2013.  LCS  5  through  LCS  12 
are  currently  in  various  stages  of  construction.  The 
Navy  is  concerned  about  both  contractors'  ability  to 
meet  construction  schedules  without  impacting 
follow-on  hulls. 

LCS  1  completed  a  ten-month  deployment  to  the 
western  pacific  in  December  2013  where  it  operated 
out  of  Singapore.  During  this  deployment  it 
encountered  two  significant  engineering  issues  that 
significantly  curtailed  its  ability  to  get  underway:  the 
lubrication  cooling  system  ruptured  and  the  ship 
service  diesel  engine  generator  had  reliability 
issues.  In  addition  to  these  engineering  issues,  LCS 
1  had  a  number  of  combat  system  and  other 
material  failures;  including  radar  underperformance 
and  the  combat  system  unexpectedly  rebooting 
during  operations. 


Other  Program  Issues 

The  Navy  added  20  permanent  berths  to  LCS  1  to 
support  additional  manning  for  its  deployment  to 
Singapore  in  2013.  In  May  2013,  the  Navy 
determined  that  additional  permanent 
accommodations  for  a  total  crew  size  of  98  should 
be  incorporated  in  all  LCS  class  ships.  The  LCS 
program  executive  office  has  been  directed  to  add 
these  permanent  accommodations  through  either 
forward  -or  back-fitting  the  ships.  Although  the 
habitability  modification  installed  on  LCS  1  in 
support  of  its  deployment  did  not  include  the 
addition  of  increased  storage  and  water  supplies, 
the  forward  fit  installs  will  address  the  required 
services  and  auxiliary  system  modifications 
associated  with  the  installation  of  the  additional 
berthing.  Following  LCS  1's  2013  deployment,  the 
Navy  will  evaluate  lessons  learned  and  future 
manning  options.  The  Navy  expects  to  complete 
this  evaluation  in  fiscal  year  2014  and  incorporate 
any  proposed  manning  changes  beginning  in  fiscal 
year  2015  ahead  of  the  next  block  buy  decision  in 
2016,  when  24  seaframes  will  already  be  delivered, 
constructed,  or  under  contract. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  LCS  4  showed  significant 
improvement  from  LCS  2  in  level  of  completeness 
and  number  of  high  priority  trial  cards  deficiencies  at 
delivery.  Twelve  block  buy  ships  are  funded  on  the 
block  buy  contract  and  are  in  pre-production  or 
construction,  following  thorough  production 
readiness  reviews.  LCS  5  and  6  launched  in 
December  2013.  LCS  1  deployment  successfully 
validated  major  portions  of  the  LCS  concept  of 
operations  for  crew  rotation  and  contracted 
overseas  maintenance.  Ship  service  diesel 
generator  and  seawater  cooler  reliability  issues 
were  satisfactorily  addressed  during  deployment. 
Engineering  changes  have  been  incorporated  to 
prevent  and  mitigate  those  issues  in  the  future. 
Material  failures  of  the  radar  were  a  result  of  a 
procedural  error  causing  the  system  to  reboot, 
however  the  radar  performed  to  design 
specifications.  The  program  office  also  provided 
technical  comments,  which  were  incorporated 
where  deemed  appropriate. 
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Littoral  Combat  Ship  -  Mission  Packages  (LCS  MP) 


The  Navy's  Littoral  Combat  Ship  (LCS)  will  provide 
mine  countermeasures  (MCM),  surface  warfare 
(SUW),  and  antisubmarine  warfare  (ASW)  capability 
using  mission  packages.  Packages  include 
weapons  and  sensors  launched  and  recovered  from 
LCS  seaframes  and  operated  from  MH-60 
helicopters  and  unmanned  vehicles.  The  Navy 
plans  to  deliver  capability  to  the  fleet  in  increments. 
We  are  assessing  the  Navy's  progress  in  achieving 
full  baseline  performance  capability,  including  all 
planned  increments  needed  to  achieve  this 
capability  for  each  package. 


*4^ 

Surface  Warfare 

Mine  Countermeasures 

Anti-Submarine  Warfare 

(SUW) 

(MCM) 

(ASW) 

Source:  ©  Northrop  Grumman  Corporation. 
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Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Corporation  M&TS 

08/2007 

01/2014 

change 

Program  office:  Washington  Navy  Yard 

Research  and  development  cost 

$517.2 

$3,103.4 

NA 

Washington,  DC 

Procurement  cost 

$3,427.8 

$4,108.7 

19.9 

Funding  needed  to  complete: 

Total  program  cost 

$3,954.7 

$7,243.5 

NA 

R&D:  $1,647.1  million 

Program  unit  cost 

$61,792 

$113,180 

NA 

Procurement:  $3,656.4  million 

Total  quantities 

64 

64 

0.0 

Total  funding:  $5,334.9  million 

Acquisition  cycle  time  (months) 

NA 

NA 

NA 

Procurement  quantity:  51 

The  2007  RDT&E  cost  estimate  does  not  reflect  full  program  costs  and  cost  growth  is  not  applicable. 

Current  estimate  does  not  include  $3.6  billion  of  procurement  money  for  replacement  and  other  items. 

In  November  2013,  DOD  approved  the  baseline 
for  the  LCS  Mission  Packages  program  and 
previously,  in  October  2012,  transferred  decision 
authority  to  the  Navy  which  approved  the 
programs  milestone  B  decision.  The  Navy  held 
the  milestone  B  event,  which  normally  indicates 
development  start,  in  January  2014  but  has  been 
developing  the  packages  since  2004  and  has 
delivered  eight  partial  packages.  As  a  part  of  this 
event,  the  Navy  produced  the  first  full  lifecycle 
cost  estimate.  The  program’s  acquisition  cost  is 
now  $7.24  billion,  an  increase  of  $3.29  billion 
based  upon  its  2007  estimate,  which  did  not 
include  full  development  costs.  The  Navy  plans  to 
deliver  mission  package  capabilities  in 
increments,  the  MCM  and  SUW  mission 
packages  are  not  currently  expected  to  field  their 
baseline  performance  capability  until  the  Navy 
has  purchased  32  and  takes  delivery  of  24 
seaframes. 


Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  in  a  relevant 
environment 


o 


Demonstrate  all  critical  technologies  in  a  realistic 
environment 


o 


Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


Demonstrate  critical  processes  are  in  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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LCS  Packages  Program 

Mine  Countermeasures  (MCM) 

The  Navy  has  accepted  four  packages  without 
demonstrating  that  they  meet  requirements  and 
plans  to  accept  two  more  in  fiscal  2014.  The 
package  will  be  fielded  in  four  increments:  the  first 
intends  to  remove  sailors  from  the  minefield  and 
improve  mine  detection,  classification,  and 
destruction  over  legacy  vessels.  Although  DOD 
states  that  all  systems  in  increment  one  are  fully 
mature,  developmental  testing  has  shown 
performance  problems  that  led  to  changes  in 
planned  tactics,  removal  of  systems,  and  lowered 
testing  requirements.  For  example,  corrections  to 
the  mine-hunting  sonar's  reliability  have  yet  to  be 
validated  in  operational  testing,  and  two  systems, 
intended  to  sweep  for  and  neutralize  mines,  had  to 
be  removed  for  safety  and  performance  issues. 
Also,  the  Navy  now  requires  multiple  searches  to 
identify  mines,  adding  time  to  the  process,  and  has 
lowered  increment  one  testing  requirements  for 
mine  clearance  rates.  If  operational  testing  proves 
successful,  this  package  will  not  be  able  to  replace 
all  legacy  capability  until  increment  three  completion 
in  fiscal  2017. 

Surface  Warfare  (SUW) 

The  Navy  has  taken  delivery  of  four  packages,  each 
comprised  of  two  30  millimeter  guns,  as  well  as  a 
rigid  hull  inflatable  boat  prototype,  boarding  gear, 
and  armed  helicopters.  Following  the  cancellation  of 
the  non-line-of-sight  launch  system,  the  Navy 
planned  to  field  the  Griffin  missile  in  2015  as  an 
initial  capability.  However,  program  officials  stated 
that  the  Navy  is  rethinking  this  and  a  solution  is  not 
yet  known. 

Antisubmarine  Warfare  (ASW) 

The  Navy  restarted  development  of  an  ASW 
package  with  new  requirements  as  the  initial 
package  was  not  going  to  deliver  enough  capability 
over  legacy  assets.  The  Navy  is  assessing  a 
replacement,  with  initial  delivery  planned  in  2016, 
that  is  expected  to  include,  among  other 
technologies,  a  variable-depth  sonar — which, 
according  to  officials,  performed  well  in  initial 
tests — and  a  towed  array.  The  maturity  of  these 
technologies  has  not  yet  been  independently 
assessed. 


Other  Program  Issues 

The  Navy  held  a  preliminary  design  review  in  2004, 
but  the  packages  have  substantively  changed.  The 
program  held  a  milestone  B  event  in  January  2014 
with  an  estimated  acquisition  cost  of  $7.24  billion. 
Tests  for  increment  one  and  two  of  the  SUW  and 
increment  one  of  the  MCM  package  are  scheduled 
for  fiscal  2014  and  2015  respectively.  These 
assessments  evaluate  progress  in  achieving 
baseline  performance  capability,  which  includes  all 
planned  increments  for  each  package.  The  Navy 
plans  to  purchase  32  LCS  seaframes  and  take 
delivery  of  at  least  24  by  the  time  the  baseline 
performance  capabilities  of  the  MCM  and  SUW 
packages  are  proven  and  fielded. 

Program  Office  Comments 

The  Navy  states  that  our  assessment  of  program 
cost  growth  incorrectly  compares  the  acquisition 
program  baseline  against  a  fiscal  2008  baseline, 
which  does  not  reflect  the  total  acquisition.  Further, 
the  Navy  states  that  this  assessment  disregards 
near  term  operational  requirements  as  the  data 
presented  indicates  that  the  program  should  be 
delayed.  The  Navy  also  states  that  our  assertion  of 
excessive  program  risk,  due  to  concurrency,  is 
unfounded  because  developmental  testing, 
combined  with  capability  proven  during  early 
deployments,  has  significantly  reduced  technical 
risk.  This  is  evidence,  according  to  the  Navy,  that 
the  LCS  will  successfully  complete  operational 
testing.  Lastly,  the  current  missile  procurement  was 
delayed  due  to  sequestration;  the  Navy  states  that 
the  program  is  on  track  to  deliver  a  capability  in  late 
2016. 

GAO  Response 

In  comparing  the  2007  estimate  with  the  acquisition 
program  baseline,  we  used  the  Navy's  2007  data, 
which  included  full  procurement  costs  but  only  five 
years  of  development  cost.  The  Navy  has  acquired 
eight  packages  without  proving  capability  through 
operational  testing.  In  the  absence  of  a  defined 
increment-based  approach  for  the  full  baseline 
capability  to  sequentially  gain  knowledge  and  meet 
these  requirements,  the  Navy's  acquisition 
approach  is  not  in  accordance  with  best  practices  or 
DOD  guidance  in  place  at  the  time  of  our  review. 
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Mobile  User  Objective  System  (MUOS) 

The  Navy's  MUOS,  a  satellite  communication 
system,  is  expected  to  provide  a  worldwide, 
multiservice  population  of  mobile  and  fixed-site 
terminal  users  with  increased  narrowband 
communications  capacity  and  improved  availability 
for  small  terminal  users.  MUOS  will  replace  the 
Ultra  High  Frequency  (UHF)  Follow-On  (UFO) 
satellite  system  currently  in  operation  and  provide 
interoperability  with  legacy  terminals.  MUOS 
consists  of  a  network  of  satellites  and  an  integrated 
ground  network.  We  assessed  both  the  space  and 
ground  segments. 


Source:  ©2007  Lockheed  Martin. 
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Program  Essentials 

Prime  contractor:  Lockheed  Martin 
Space  Systems 
Program  office:  San  Diego,  CA 
Funding  needed  to  complete: 

R&D:  $176.9  million 
Procurement:  $975.1  million 
Total  funding:  $1,152.0  million 
Procurement  quantity:  1 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 

As  of 
12/2004 

$3,836.4 

Latest 

09/2013 

$4,511.6 

Percent 

change 

17.6 

Procurement  cost 

$3,192.1 

$2,867.4 

-10.2 

Total  program  cost 

$7,069.1 

$7,448.1 

5.4 

Program  unit  cost 

$1,178,186 

$1,241,345 

5.4 

Total  quantities 

6 

6 

0.0 

Acquisition  cycle  time  (months) 

90 

NA 

NA 

Latest  acquisition  cycle  time  could  not  be  calculated  because  the  most  recent  MUOS  program 
baseline  does  not  estimate  dates  for  operational  capability. 


The  MUOS  program's  critical  technologies  are 
mature,  its  design  is  stable,  and  according  to  the 
program  office,  its  manufacturing  process 
maturity  has  increased.  The  first  satellite  was 
launched  in  February  201 2 — 26  months  later  than 
planned  at  development  start — and  the  second 
satellite  was  launched  in  July  2013.  Subsequent 
launches  of  MUOS  satellites  remain  important 
due  to  the  past  operational  failures  of  two  UFO 
satellites  and  predicted  end-of-life  of  on-orbit  UFO 
satellites,  one  of  which  was  taken  off-line  in 
November  2012.  A  remaining  challenge  is  that 
users  will  not  be  able  to  utilize  advanced  MUOS 
capabilities  in  large  part  because  of  delays  in  the 
development,  testing,  and  fielding  of  new  user 
terminals.  Current  MUOS  user  terminal 
procurement  and  fielding  are  managed  by  the 
Army's  Tactical  Radios  Project  Office. 
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MUOS  Program 

Technology,  Design,  and  Production  Maturity 

The  MUOS  program's  technologies  are  mature,  its 
design  is  stable,  and  its  manufacturing  process 
maturity  has  increased.  The  first  two  satellites  have 
been  launched  and  three  other  satellites  are  being 
built.  We  could  not  assess  whether  critical 
manufacturing  processes  were  in  control  as  the 
program  does  not  collect  statistical  process  control 
data.  The  program  has  experienced  quality 
problems  in  the  past  that  resulted  in  cost  increases 
and  schedule  delays;  however,  the  number  of 
manufacturing  defects  on  the  space  segment  has 
decreased  over  time.  According  to  the  program, 
multiple  corrective  action  boards  collect  and  track  all 
defects  in  manufacturing  processes  and  the 
program  uses  this  data  to  assess  the  maturity  of 
production.  Examples  of  specific  metrics  used 
include  number  of  defects  per  1 ,000  hours  of  touch 
labor,  amount  of  deferred  work  and  associated  risk, 
and  rate  of  resolving  nonconformance  issues. 
According  to  the  program  office,  these  metrics 
indicate  continuing  increases  in  MUOS  production 
maturity  and  show  program  goals  are  being  met. 

Other  Program  Issues 

The  first  MUOS  satellite  was  launched  in  February 
2012,  26  months  later  than  initially  planned,  and  the 
second  satellite  was  launched  in  July  2013. 
According  to  the  program  office,  the  remaining 
MUOS  satellites  are  needed  because  most  on-orbit 
UFO  satellites  are  past  their  design  lives.  Two  of 
these  unexpectedly  failed — one  in  June  2005  and 
another  in  September  2006 — and  a  third  was  taken 
off-line  in  November  2012.  Program  officials  stated 
that  the  required  availability  level  of  legacy  UHF 
communication  capabilities  has  been  provided  and 
based  on  the  projected  launch  schedule  for  the 
remaining  MUOS  satellites,  the  required  availability 
of  legacy  UHF  communication  capabilities  is 
predicted  to  be  maintained  through  May  2020. 

Since  2007,  we  have  reported  that  synchronizing 
deliveries  of  MUOS  satellites  with  compatible  Joint 
Tactical  Radio  System  (JTRS)  Handheld,  Manpack, 
and  Small  Form  Fit  (HMS)  terminals  has  been  a 
challenge.  Launching  MUOS  satellites  is  important 
to  sustain  legacy  UHF  communications  capability. 
However,  utilization  of  over  90  percent  of  MUOS's 
planned  capability  is  dependent  on  the  development 
of  the  MUOS  waveform — which  completed  formal 


qualification  testing  in  November  201 2 — and  porting 
the  MUOS  capability  onto  operational  user 
terminals.  The  Army's  HMS  program  is  developing 
the  first  operational  terminal  to  incorporate  the 
MUOS  waveform.  Additionally,  according  to  the 
MUOS  program,  other  vendors  wanting  to  integrate 
the  MUOS  waveform  into  their  radios  are  being 
supported  with  MUOS  waveform  software  and 
government-operated  MUOS  test  resources. 

To  date,  the  HMS  program  has  conducted  multiple 
developmental  tests  on  the  Manpack  radio  and  an 
operational  test  was  conducted  in  2012.  This 
testing,  conducted  without  the  MUOS  waveform, 
found  that  the  terminals  did  not  meet  all 
performance  and  reliability  requirements.  However, 
according  to  the  MUOS  program,  the  radio 
demonstrated  improved  performance  at  subsequent 
test  events  and  was  selected  for  fielding  to  Army 
units  without  the  MUOS  waveform  due  to  the 
capability  increases  it  would  provide.  Limited  testing 
has  been  conducted  on  terminals  with  the  MUOS 
waveform  to  date.  Operational  testing  and  initial 
fielding  of  the  HMS  terminals  with  the  MUOS 
waveform  is  planned  to  begin  in  2014. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  officials  provided  technical  comments, 
which  were  incorporated  as  appropriate. 
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MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (MQ-1C  Gray  Eagle) 

The  Army's  MQ-1C  Gray  Eagle  unmanned  aircraft 
system  (UAS)  will  perform  reconnaissance, 
surveillance,  target  acquisition,  and  attack  missions 
either  alone  or  with  other  platforms  such  as  the 
Longbow  Apache  helicopter.  Each  platoon-sized 
system  includes  four  aircraft,  payloads,  data 
terminals,  automatic  take  off  and  landing  systems, 
ground  control  stations,  and  support  equipment. 

The  Army  procured  less-capable  quick  reaction 
capability  variants  to  address  urgent  needs,  but 
these  systems  are  not  included  as  part  of  the  Gray 
Eagle  program. 

Source:  General  Atomics  Aeronautical  Systems,  Inc. 
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Program  Essentials 

Prime  contractor:  General  Atomics 
Aeronautical  Systems,  Inc. 

Program  office:  Redstone  Arsenal,  AL 
Funding  needed  to  complete: 

R&D:  $76.8  million 
Procurement:  $849.5  million 
Total  funding:  $1,115.4  million 
Procurement  quantity:  6 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 
Procurement  cost 
Total  program  cost 
Program  unit  cost 
Total  quantities 

Acquisition  cycle  time  (months) 


As  of 

Latest 

Percent 

04/2005 

08/2013 

change 

$362.7 

$1,006.5 

177.5 

$705.1 

$3,227.8 

357.8 

$1,067.8 

$4,859.2 

355.1 

$213,552 

$156,750 

-26.6 

5 

31 

520.0 

50 

95 

90.0 

Total  quantities  include  31  platoon  systems  consisting  of  four  aircraft  and  related  equipment.  The 
program  will  also  buy  21  aircraft  to  replace  attrition  losses  and  7  training  aircraft,  for  a  total  of  152. 


The  Gray  Eagle  program  received  approval  to 
begin  full-rate  production  in  June  2013  with  its 
technologies,  design,  and  manufacturing 
processes  mature.  However,  design  changes  and 
supplier  base  issues  put  the  program's  cost  and 
schedule  at  risk.  Program  officials  said  they  are 
considering  a  change  to  the  aircraft  tail,  which 
would  be  costly  and  require  retrofitting  the  entire 
fleet.  The  program  is  also  developing  a  new 
ground  control  station  which  will  not  undergo 
operational  testing  until  May  2015.  In  addition,  a 
production  readiness  review  conducted  in  support 
of  the  program's  full-rate  production  decision 
identified  several  high  risk  supplier  base  issues 
that  pose  uncertainty  for  the  program's  cost  and 
schedule.  The  program  is  tracking  these  risks  and 
program  officials  said  its  suppliers  have  mitigation 
strategies  in  place. 
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MQ-1C  Gray  Eagle  Program 

Technology  and  Design  Maturity 

The  Gray  Eagle  program  reported  that  all  five  of  its 
critical  technologies  are  mature  and  have  been 
demonstrated  in  a  combat  environment.  Program 
officials  stated  that  the  design  is  generally  stable; 
however,  there  is  the  potential  for  design  changes  if 
the  program  decides  to  integrate  a  new  aircraft  tail 
that  the  program's  contractor  is  developing  in 
response  to  an  aircraft  accident  in  2009.  Program 
officials  said  that  integrating  the  new  tail  design 
would  be  costly  as  it  would  require  retrofitting  the 
entire  Gray  Eagle  fleet,  although  it  would  provide 
increased  reliability  and  endurance.  A  decision  is 
expected  in  fiscal  year  2014  on  whether  to  move 
forward  with  the  retrofit  based  on  affordability 
considerations. 

Production  Maturity 

The  Gray  Eagle's  2013  production  readiness 
assessment  reported  that  its  manufacturing 
processes  are  mature  and  at  the  level 
recommended  by  DOD  guidance  for  the  start  of  full- 
rate  production.  The  program  was  approved  to 
begin  full-rate  production  of  49  aircraft  and  support 
equipment  in  June  2013  and  awarded  the  first  of 
three  planned  full-rate  production  contracts  in 
September  2013.  However,  the  program  is  tracking 
several  risks  related  to  its  supplier  base.  For 
example,  the  Gray  Eagle  prime  contractor  is 
currently  flight  testing  a  new  aircraft  engine  to 
replace  the  legacy  engine  because  the  legacy 
engine  supplier  went  into  bankruptcy.  Program 
officials  noted  they  have  enough  engines  on  hand 
for  the  first  full-rate  production  lot,  identifying  a  new 
engine  source  poses  technical,  cost,  and  schedule 
risks  to  the  program,  according  to  the  production 
readiness  review.  The  program  and  the  Defense 
Contract  Management  Agency  are  also  tracking 
other  risk  items  related  to  multiple  suppliers' 
financial  concerns  as  well  as  quality  control,  but 
program  officials  said  its  suppliers  have  mitigation 
strategies  to  address  these  risks. 

Other  Program  Issues 

The  Gray  Eagle  system  completed  initial 
operational  test  and  evaluation  in  August  2012. 
DOD's  Director,  Operational  Test  and  Evaluation 
(DOT&E),  found  the  system  to  be  operationally 
effective  with  the  potential  to  provide  effective 
support  to  combat  units  and  suitable  at  meeting  its 


performance  and  combat  availability  requirements. 
However,  several  recommended  improvements 
were  identified  related  to  tactics,  techniques,  and 
procedures;  training;  and  the  design  of  the  ground 
control  station  shelter.  Specifically,  DOT&E  found 
that  the  Army  needs  to  improve  integration  of  Gray 
Eagle  into  employment  concepts  and  improve 
soldier  training.  In  addition,  the  design  of  the  current 
ground  control  station  degrades  aircraft  operator 
efficiency.  According  to  the  program  office,  the  Army 
has  already  begun  to  address  these  issues  and 
plans  to  re-evaluate  them  at  the  planned  follow-on 
test  and  evaluation  in  May  2015.  The  follow-on  test 
will  also  be  used  to  assess  new  hardware  and 
software  the  program  is  developing  to  replace  the 
current  ground  control  station  as  well  as  assess  its 
net  readiness  capabilities.  Once  testing  is  complete, 
program  officials  said  the  Army  will  deploy  the  new 
ground  control  station  to  fielded  units  and  will  train 
Gray  Eagle  operators  on  the  new  system. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 


The  Navy's  MQ-4C  Triton  is  intended  to  provide  a 
persistent  maritime  intelligence,  surveillance,  and 
reconnaissance  (ISR)  capability  even  when  no 
other  naval  forces  are  present.  Triton  will  operate 
from  five  land-based  sites  worldwide  as  a  part  of  a 
family  of  maritime  patrol  and  reconnaissance 
systems  that  recapitalizes  the  Navy's  airborne  ISR 
assets.  Planned  improvements  include  a  signals 
intelligence  capability  and  an  upgrade  to  the 
systems  communication  relay.  The  Triton  is  based 
on  the  Air  Force's  RQ-4B  Global  Hawk  air  vehicle. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Systems  Corporation 

02/2009 

12/2012 

change 

Program  office:  Patuxent  River,  MD 

Research  and  development  cost 

$3,304.2 

$3,639.0 

10.1 

Funding  needed  to  complete: 

Procurement  cost 

$9,805.7 

$9,592.7 

-2.2 

R&D:  $770.8  million 

Total  program  cost 

$13,512.2 

$13,544.2 

0.2 

Procurement:  $9,542.6  million 

Program  unit  cost 

$193,032 

$193,488 

0.2 

Total  funding:  $10,519.3  million 

Total  quantities 

70 

70 

0.0 

Procurement  quantity:  65 

Acquisition  cycle  time  (months) 

92 

109 

18.4 

The  Triton's  critical  technology  is  mature,  the 
design  is  stable — although  not  yet 
demonstrated — and  the  program  reports  that 
manufacturing  processes  will  be  ready  to  support 
low-rate  production.  Testing  on  the  first 
development  aircraft  began  in  September  2012 
and  first  flight  occurred  in  May  2013.  However, 
the  program  discovered  design  challenges  during 
integration  and  testing  and  had  its  production 
budget  reprioritized  in  fiscal  year  2014. 
Alternatives  for  an  air-to-air  radar  subsystem  are 
being  considered  and  the  program's  production 
decision  has  been  delayed  by  about  one  year. 
The  program  office  authorized  the  prime 
contractor  to  develop  a  new  target  baseline  and 
schedule  that  takes  into  account  the  program's 
increased  costs  and  schedule  delays. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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MQ-4C  Triton  Program 

Technology  and  Design  Maturity 

The  Triton  program's  one  critical  technology — a 
hydrocarbon  sensor — was  found  to  be  mature  by  an 
April  2011  technology  readiness  assessment. 
However,  the  air-to-air  radar  subsystem  (AARSS), 
which  will  enable  Triton  to  sense  and  avoid  other 
aircraft,  has  faced  developmental  delays  and  cost 
overruns.  As  a  result,  the  Navy  ordered  work  to  be 
stopped  on  the  subsystem  in  April  2013  and  is 
currently  examining  alternatives. 

The  Triton  design  is  stable  and  according  to  the 
program  office,  all  design  drawings  are  releaseable 
to  manufacturing,  but  a  fully  outfitted  system-level 
prototype  has  not  yet  been  tested.  Testing  of  the 
first  development  aircraft  began  in  September  201 2, 
followed  by  first  flight  in  May  201 3.  Program  officials 
expect  the  second  development  aircraft,  which  was 
expected  to  be  the  first  with  a  full  sensor  suite  and  a 
sense-and-avoid  capability,  to  begin  testing  in  2014. 
However,  as  a  result  of  developmental  delays,  they 
do  not  plan  for  AARSS  to  be  incorporated  on  the 
aircraft  until  2016  for  operational  evaluation. 
Software  development  and  maturity  also  remain 
areas  of  potential  risk.  The  program  plans  to  utilize 
nearly  8  million  lines  of  code,  more  than  20  percent 
of  which  will  be  new.  Though  some  software 
integration  challenges  discovered  during  testing 
were  resolved  in  201 3,  two  more  phases  of  software 
development  and  installation  are  scheduled  for 
completion  before  the  aircraft  enters  into 
operational  evaluation. 

Production  Maturity 

In  November  2011,  the  program  received  approval 
to  build  three  air  vehicles  and  ground  stations,  in 
part  to  demonstrate  its  manufacturing  processes 
prior  to  production.  Program  officials  noted  that  one 
of  these  three  aircraft  will  no  longer  be  produced 
because  of  new  priorities  within  the  Navy's  budget. 
The  program  reports  that  manufacturing  processes 
will  be  able  to  support  low  rate  initial  production 
after  the  production  decision  is  made.  The  Triton 
aircraft  is  based  on  the  Air  Force's  RQ-4B  Global 
Hawk  and  uses  sensor  components  and 
subsystems  from  other  platforms.  There  are  some 
structural  changes  to  the  airframe,  but  none  of 
these  require  significant  changes  to  manufacturing 


processes.  The  program  plans  to  award  a  contract 
for  long  lead  material  one  year  prior  to  the  low-rate 
initial  production  contract  award. 

Other  Program  Issues 

The  Triton  program's  production  decision  has  been 
delayed  by  about  one  year  because  of  design 
adjustments  to  ensure  airworthiness  and  software 
stability  issues  discovered  during  system  integration 
and  testing  as  well  as  the  Navy's  decision  to 
reprioritize  the  program's  production  funding  for 
fiscal  year  2014.  The  program  office  authorized  the 
prime  contractor  to  develop  a  new  target  baseline 
and  schedule  to  take  into  account  increased  costs 
and  delays.  Program  officials  expected  to  receive 
the  revised  baseline  and  schedule  in  December 
2013.  According  to  program  officials,  the  delay  has 
allowed  the  contractor  to  resolve  some  technical 
issues. 

In  September  2013,  the  Global  Hawk  program 
experienced  an  anomaly  with  a  navigation  system 
held  in  common  with  Triton.  Though  the  Triton 
program  did  not  experience  the  same  anomaly,  test 
flights  were  suspended  until  a  work  around  was 
identified.  The  Air  Force  is  continuing  its 
investigation  into  the  cause  of  the  navigation  system 
issue. 

Program  Office  Comments 

In  addition  to  providing  technical  comments,  the 
program  office  noted  that  the  Triton  UAS  program 
continues  to  demonstrate  success  during  its  system 
development  and  demonstration  phase  as 
evidenced  by  the  nine  initial  envelope  expansion 
flights  conducted  to  date.  As  reported  in  the 
December  201 2  Selected  Acquisition  Report,  the 
program  breached  the  acquisition  program  baseline 
cost  threshold  for  research,  development,  test  and 
evaluation  and  the  schedule  thresholds  for  reaching 
production  start,  operational  evaluation  start,  full 
rate  production,  and  initial  operational  capability. 
The  program  has  been  re-planned  to  adjust  the  cost 
and  schedule  remaining  for  the  system 
development  and  demonstration  contract.  A 
proposed  program  baseline  revision  is  in  work  and 
all  parameters  will  be  reviewed,  to  include  the 
operating  and  support  cost  estimate.  The  program 
continues  to  benefit  from  strong  support  within  the 
Department  of  the  Navy. 
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MQ-8  (Fire  Scout) 

The  Navy's  MQ-8  unmanned  aerial  vehicle  is 
intended  to  provide  real-time  imagery  and  data  in 
support  of  intelligence,  surveillance,  and 
reconnaissance  missions.  An  MQ-8  system  is 
comprised  of  up  to  three  air  vehicles  with  sensors, 
two  ground  control  stations,  and  one  recovery 
system.  The  air  vehicle  launches  and  lands 
vertically,  and  operates  from  ships  and  land.  The 
MQ-8  is  intended  for  use  in  various  operations, 
including  surface,  anti-submarine,  and  mine 
warfare.  We  assessed  the  latest  variant,  the  MQ- 
8C. 


Source:  Northrop  Grumman  Corporation. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Patuxent  River,  MD 

12/2006 

09/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$629.0 

$711.8 

13.2 

R&D:  $0.0  million 

Procurement  cost 

$1,769.6 

$2,099.4 

18.6 

Procurement:  $1,685.1  million 

Total  program  cost 

$2,750.7 

$2,811.2 

2.2 

Total  funding:  $1,685.1  million 

Program  unit  cost 

$15,540 

$16,064 

3.4 

Procurement  quantity:  145 

Total  quantities 

177 

175 

-1.1 

Acquisition  cycle  time  (months) 

104 

169 

62.5 

The  Navy  ceased  production  of  the  MQ-8B 
variant  and  in  April  2012  awarded  a  contract  to 
begin  development  and  production  of  the  new 
MQ-8C  with  a  larger  airframe  to  improve  range, 
endurance,  payload,  and  future  carrying  capacity. 
MQ-8C  relies  on  common,  mature  technologies 
with  the  MQ-8B.  The  engineering  design  of  the 
MQ-8C  is  complete  as  it  is  based  on  the  MQ-8B 
design,  which  appeared  to  be  stable  before 
halting  production.  The  program  completed 
operational  test  and  evaluation  of  MQ-8B  in 
December  2013  and  a  Quick  Reaction 
Assessment  of  MQ-8C  will  be  completed  in  the 
fourth  quarter  of  fiscal  year  2014.  The  program 
plans  to  conduct  an  acquisition  strategy  review  in 
the  first  quarter  of  fiscal  year  2014  that  assesses 
overall  program  health,  including  production 
readiness.  In  2015,  the  program  plans  to  test  the 
MQ-8C  on  the  Littoral  Combat  Ship,  the  platform 
originally  intended  for  MQ-8B. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line  % 

•  Test  a  production-representative  prototype  0 

9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Fire  Scout  Program 

Technology  and  Design  Maturity 

According  to  the  program  office,  MQ-8C  is  90 
percent  common  with  the  previously  developed  MQ- 
8B,  with  the  primary  difference  being  structural 
modifications  to  accommodate  the  MQ-8C's  larger 
airframe  and  fuel  system.  The  MQ-8C  relies  on 
mature  technologies  common  to  the  MQ-8B  and 
does  not  introduce  any  new  technologies.  Program 
officials  stated  that  the  engineering  design  of  MQ- 
8C  is  complete,  as  it  is  based  on  the  MQ-8B  design, 
which  appeared  to  be  stable  prior  to  the  Navy's 
decision  to  cease  production  of  the  system  in  2012. 
Program  officials  noted  that  a  revision  to  the  air 
vehicle  to  accommodate  increased  payload  remains 
to  be  completed,  but  they  expect  the  issue  to  be 
resolved  by  early  calendar  year  2014.  The  first  flight 
of  the  MQ-8C  occurred  in  October  2013. 

Production  Maturity 

The  program  plans  to  produce  the  MQ-8C  on  the 
same  production  line  as  the  MQ-8B  and  officials 
stated  that  they  have  not  experienced  challenges  in 
transitioning  production  between  the  systems.  The 
program's  production  processes  for  MQ-8B  have 
been  demonstrated,  but  we  could  not  assess 
whether  critical  manufacturing  processes  were  in 
control  as  the  program  does  not  collect  data  on 
statistical  process  controls  or  assess  process 
capabilities  using  manufacturing  readiness  levels. 
The  program  suspended  production  of  MQ-8B  and 
awarded  a  development  and  production  contract  for 
8  MQ-8Cs  in  April  2012.  The  program  earlier 
intended  to  resume  production  of  the  MQ-8B  aircraft 
in  2015  to  align  with  the  delivery  of  the  Littoral 
Combat  Ship — the  platform  from  which  MQ-8B  is 
intended  to  operate — but  the  Navy  plans  to  continue 
MQ-8C  production  instead.  The  Navy  plans  to 
award  a  third  production  contract  for  MQ-8C  in 
March  2014  in  support  of  the  Joint  Emergent 
Operational  Need.  Program  officials  stated  that  the 
date  of  the  full-rate  production  decision  has  not 
been  determined.  Future  procurements  will  be 
addressed  in  the  updated  acquisition  strategy. 

Other  Program  Issues 

The  program  has  shifted  its  focus  to  MQ-8C — an 
upgraded  variant  of  the  aircraft  with  improved 
range,  endurance,  payload,  and  future  carrying 
capacity — due  to  a  joint  urgent  operational  need 
from  U.S.  Africa  Command.  According  to  program 


officials,  the  program  is  not  requesting  new  funds  for 
MQ-8C,  but  plans  to  use  funding  slated  for  MQ-8B. 
MQ-8C  is  not  required  to  conform  to  many  of  the 
acquisition  practices  followed  by  other  programs  as 
it  is  being  developed  to  fill  a  joint  urgent  operational 
need.  However,  a  Quick  Reaction  Assessment  is 
planned  for  MQ-8C  3  to  4  months  prior  to  ship 
deployment,  which  is  expected  to  be  in  the  first 
quarter  of  fiscal  year  2015.  The  program  is  planning 
to  test  the  MQ-8C  at-sea  in  2014  on  the  DDG-109 
and  on  the  Littoral  Combat  Ship  in  2015.  The 
program  completed  operational  test  and  evaluation 
of  MQ-8B  in  December  2013  and  a  Quick  Reaction 
Assessment  of  MQ-8C  will  be  completed  in  the 
fourth  quarter  of  fiscal  year  2014.  The  program 
plans  to  conduct  an  acquisition  strategy  review  in 
the  first  quarter  of  fiscal  year  2014  that  assesses 
overall  program  health,  including  production 
readiness. 

Program  Office  Comments 

According  to  the  program  office,  the  MQ-8  Fire 
Scout  system  has  accumulated  over  4,500  flights 
totaling  more  than  12,000  flight  hours  with  the  MQ- 
8B  air  vehicle.  In  the  first  quarter  of  fiscal  year  2014, 
the  system  was  assessed  by  the  Navy  Operational 
Test  community  aboard  USS  Roberts  and 
completed  an  Initial  Operational  Capability 
Supportability  Review  in  preparation  for  fleet 
transition  and  Initial  Operational  Capability  in 
second  quarter  fiscal  year  2014.  The  Fire  Scout 
system  with  the  MQ-8C  Endurance  Upgrade  aircraft 
is  being  procured  in  response  to  a  Joint  Emergent 
Operational  Need  from  Africa  Command.  The  Navy 
is  planning  to  incorporate  the  MQ-8C  variant  into 
the  Fire  Scout  system  as  part  of  its  updated 
acquisition  strategy.  First  flight  of  the  MQ-8C 
occurred  on  October  31,  2013.  As  of  January  2014, 
the  MQ-8C  has  performed  more  than  32  test  flights 
and  accumulated  over  46  flight  hours.  In 
commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

The  Air  Force's  MQ-9  Reaper  is  a  multirole, 
medium-to-high-altitude  endurance  unmanned 
aircraft  system  capable  of  flying  at  higher  speeds 
and  altitudes  than  its  predecessor,  the  MQ-1 
Predator  A.  The  Reaper  is  designed  to  provide  a 
ground-attack  capability  to  find,  fix,  track,  target, 
engage,  and  assess  small  ground  mobile  or  fixed 
targets.  Each  system  consists  of  four  aircraft,  a 
ground  control  station,  and  a  satellite 
communications  suite.  We  assessed  the  increment  I 
Block  5  configuration  and  made  observations  on 
Block  1. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  General  Atomics 

As  of 

Latest 

Percent 

Aeronautical  Systems,  Inc. 

02/2008 

09/2013 

change 

Program  office:  Wright-Patterson  AFB, 

Research  and  development  cost 

$441.9 

$1,613.0 

265.0 

OH 

Procurement  cost 

$2,220.7 

$10,826.4 

387.5 

Funding  needed  to  complete: 

Total  program  cost 

$2,773.5 

$12,517.3 

351.3 

R&D:  $843.3  million 

Program  unit  cost 

$26,414 

$30,983 

17.3 

Procurement:  $6,055.7  million 

Total  quantities 

105 

404 

284.8 

Total  funding:  $6,921.1  million 

Acquisition  cycle  time  (months) 

79 

100 

26.6 

Procurement  quantity:  161 

The  Reaper's  critical  technologies  are  mature  and 
all  design  drawings  have  been  released.  Delays 
in  manufacturing,  delivery  of  technical  data,  and 
developmental  testing  have  pushed  back  critical 
software  releases  for  both  Block  1  and  Block  5 
configurations.  The  MQ-9  Reaper  was  approved 
for  low-rate  initial  production  in  November  2012, 
but  has  not  yet  demonstrated  that  critical 
manufacturing  processes  are  in  control.  The 
program  originally  expected  to  complete 
operational  testing  no  later  than  October  2014; 
however  manufacturing  and  other  delays  have 
caused  the  program  to  postpone  the  completion 
of  testing  until  January  2016.  The  program's  full- 
rate  production  decision  was  changed  to  an  in- 
process  review  because  more  than  half  of  the 
aircraft  will  be  procured  prior  to  a  full-rate 
production  decision. 


•  Demonstrate  critical  processes  are  in  control  O 


•  Demonstrate  critical  processes  on  a  pilot  production  line  o 

•  Test  a  production-representative  prototype  £ 

9  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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MQ-9  Reaper  Program 

Technology  and  Design  Maturity 

The  Reaper's  critical  technologies  are  considered 
mature  and  its  design  is  stable.  Manufacturing 
issues,  as  well  as  delays  in  the  delivery  of  technical 
data  and  developmental  testing  have  pushed  back 
critical  Block  1  and  Block  5  configuration  software 
releases  by  8  months  and  15  months,  respectively. 
Program  officials  now  plan  to  field  the  Block  5 
software  in  March  2016.  The  Block  5  configurations 
include  capability  enhancements  from  the  Block  1 
configuration  such  as  updates  to  the  radar,  data 
link,  sensor,  landing  gear,  software,  and  ground 
control  stations.  Until  testing  is  completed,  the  Block 
5  system  remains  at  risk  of  design  changes. 

Production  Maturity 

The  Air  Force  approved  low-rate  initial  production  of 
the  MQ-9  Reaper  Block  5  Increment  in  November 
2012.  According  to  program  officials,  the  Block  5 
manufacturing  processes  have  reached  the  level  of 
maturity  recommended  by  DOD  guidance  for  the 
start  of  low-rate  production.  However,  the  program's 
manufacturing  readiness  level  did  not  indicate  that 
its  production  processes  were  in  statistical  control, 
as  recommended  by  our  best  practices.  MQ-9 
Reaper  production  deliveries  have  slowed  over  the 
past  year  because  of  delinquencies  in  completing 
technical  data,  software  delays,  and  fuel  tank  issues 
that  required  engineering  design  changes  to  the 
production  line  and  retrofitting  aircraft  that  had 
already  been  produced.  As  of  December  2013,  21 
Block  1  aircraft  have  been  produced,  but  are  still 
awaiting  the  necessary  software  capability 
upgrades  before  they  can  be  delivered.  Until  these 
software  upgrades  are  complete,  aircraft  are  only 
being  delivered  based  on  urgent  needs.  According 
to  program  officials,  the  program  has  developed  an 
aircraft  delivery  recovery  plan  that  should  allow 
deliveries  to  be  back  on  track  by  April  2014. 

Other  Program  Issues 

The  program  office  declared  a  breach  to  the  MQ-9 
Block  5  program's  schedule  baseline  in  May  2013 
because  of  delays  associated  with  manufacturing, 
technical  data,  and  developmental  testing.  The 
program  originally  expected  to  complete  operational 
testing  no  later  than  October  2014.  Operational 
testing  will  now  be  complete  in  January  2016.  This 
schedule  change  was  approved  by  the  configuration 
steering  board  in  December  2013.  At  this  time,  the 


full-rate  production  decision  was  changed  to  an  in- 
process  review  since  more  than  half  of  the  aircraft 
will  be  procured  prior  to  the  full  rate  production 
decision.  The  program  office  is  in  the  process  of 
updating  the  acquisition  program  baseline. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assesment, 
program  officials  noted  that  since  the  program  office 
declared  a  breach  in  May  2013,  the  program  office 
and  contractor  resolved  issues  that  delayed 
schedule  and  are  implementing  processes  to 
improve  quality  for  future  success.  The  re-baselined 
MQ-9  Block  5  schedule  was  approved  at  the  Air 
Force  review  board  in  August  2013  and  the 
configuration  steering  board  in  December  2013. 
The  program  office  is  updating  the  acquisition 
program  baseline  to  reflect  program  changes. 
Follow-on  test  and  evaluation  is  now  planned  to 
complete  in  January  2016.  The  program  office  is 
currently  in  formal  developmental  test  of  Block  5. 
The  program  office  also  took  delivery  of  the  first 
Block  5  aircraft  in  November  2013  to  assist  with 
developmental  test  and  technical  order 
development.  The  program  office  also  provided 
technical  comments  that  were  incorporated  as 
appropriate. 
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Next  Generation  Operational  Control  System  (GPS  OCX) 


The  Air  Force's  Global  Positioning  System  Next 
Generation  Operational  Control  System  (GPS  OCX) 
is  to  replace  the  current  ground  control  system  for 
legacy  and  new  GPS  III  satellites.  GPS  OCX  is 
expected  to  ensure  reliable  and  secure  delivery  of 
position  and  timing  information  to  military  and 
civilian  users.  The  Air  Force  plans  to  develop  GPS 
OCX  in  blocks,  with  each  block  delivering  upgrades 
as  they  become  available.  We  assessed  the  initial 
three  blocks,  which  support  the  launch,  checkout, 
and  operation  of  GPS  II  and  III  satellites. 


Source:  U.S.  Air  Force. 
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Program  Essentials 

Prime  contractor:  Raytheon 
Program  office:  El  Segundo,  CA 
Funding  FY14  to  FY19: 

R&D:  $1,662.8  million 
Procurement:  $0.0  million 
Total  funding:  $1,662.8  million 
Procurement  quantity:  0 


Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Latest 

Percent 

As  of 

12/2012 

change 

Research  and  development  cost 

NA 

$3,470.3 

NA 

Procurement  cost 

NA 

$0.0 

NA 

Total  program  cost 

NA 

$3,470.3 

NA 

Program  unit  cost 

NA 

$3,470,350 

NA 

Total  quantities 

NA 

1 

NA 

Acquisition  cycle  time  (months) 

Table  includes  costs  through  fiscal  year  2019. 

NA 

55 

NA 

GPS  OCX  entered  system  development  in 
November  2012.  The  program  consists  of  three 
blocks:  Block  0,  intended  to  support  the  launch 
and  checkout  of  GPS  III  satellites;  Block  1, 
designed  for  command  and  control  of  the  GPS  II 
and  III  satellites:  and  Block  2,  which  is  expected 
to  support,  monitor,  and  control  additional 
navigation  signals.  The  program's  14  critical 
technologies  are  nearing  maturity,  but  seven  of 
those  technologies  are  not  expected  to  reach  full 
maturity  until  the  end  of  fiscal  year  2015  or  later, 
nearly  3  years  after  the  program  began  system 
development.  Aligning  the  schedules  for  GPS 
OCX  and  GPS  III  is  a  risk  for  both  programs. 
Block  1  is  scheduled  to  become  operational  by 
October  2016,  6  months  after  the  date  of  GPS  Ill's 
availability  for  launch. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

Q  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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GPS  OCX  Program 

Technology  Maturity 

GPS  OCX  entered  system  development  in 
November  201 2  with  all  of  its  1 4  critical 
technologies  tested  in  a  relevant  environment,  but 
not  a  realistic  environment.  According  to  program 
officials,  these  technologies  will  not  reach  full 
maturity  until  the  program  integrates  together  all  of 
the  key  subsystems  and  components  and  tests 
them  in  the  factory.  The  first  of  these  tests  is 
scheduled  to  occur  near  the  end  of  fiscal  year  2014 
and  is  expected  to  result  in  seven  of  the 
technologies  reaching  full  maturity.  The  remaining 
seven  technologies  are  not  expected  to  reach  full 
maturity  until  the  end  of  fiscal  year  2015  or  later, 
nearly  three  years  after  the  program  began  system 
development. 

Design  and  Production  Maturity 

GPS  OCX  is  primarily  a  software  development 
program  and  is  expected  to  provide  capabilities  in 
three  block  deliverables.  Block  0  is  intended  to 
provide  initial  capability  for  launch  control  and 
checkout  of  GPS  III  satellites.  Block  1  is  designed  to 
operate  legacy  and  new  GPS  satellites,  including 
new  civilian  signals,  and  Block  2  is  expected  to  add 
operational  control  of  new  international  and 
modernized  military  signals. 

According  to  the  program  office,  software  is  being 
developed  and  tested  incrementally  with  a  critical 
design  review  after  each  stage  to  ensure  the  design 
is  stable.  Software  development  for  Block  0  is  nearly 
complete  and  all  remaining  testing  is  scheduled  for 
completion  at  the  beginning  of  fiscal  year  201 5.  The 
program  has  started  development  activities  for 
Block  1 ,  which  program  officials  expect  to  be 
completed  in  October  201 6. 

Program  officials  expect  to  integrate  the  software 
and  hardware  components  for  all  of  Blocks  0  and  1 
and  test  them  as  a  prototype  to  ensure  the  system 
will  work  as  intended.  According  to  GAO's  best 
practices,  this  integrated  prototype  test  should 
occur  before  critical  design  review.  However, 
program  officials  stated  that  the  test  will  not  happen 
until  December  2015,  18  months  after  the  final 
critical  design  review  and  ten  months  before  Block  1 
is  scheduled  to  be  complete.  Program  officials 
reported  that  they  had  used  other  knowledge-based 
practices  to  increase  confidence  in  the  stability  of 


the  design  such  as  identifying  key  product 
characteristics  and  establishing  a  reliability  growth 
curve. 

Other  Program  Issues 

The  program  experienced  significant  schedule 
delays  while  in  technology  development.  This, 
combined  with  the  program's  late  start  relative  to 
GPS  III,  has  resulted  in  considerable  risk  in  aligning 
the  schedules  of  GPS  OCX  and  GPS  III  satellites. 
Block  1,  which  is  to  provide  command  and  control 
for  GPS  III  satellites  on  orbit,  is  not  scheduled  to 
become  operational  until  October  2016,  6  months 
after  the  first  GPS  planned  GPS  III  satellite  is 
expected  to  be  available  for  launch.  Until  Block  1  is 
operational,  the  GPS  III  satellite  will  have  very 
limited  capabilities  and  Air  Force  officials  said  they 
do  not  plan  to  launch  a  second  GPS  III  satellite  until 
that  time. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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P-8A  Poseidon  Multi-Mission  Maritime  Aircraft  (P-8A) 


The  Navy's  P-8A  Poseidon  is  a  commercial 
derivative  aircraft  that  will  replace  the  P-3C  Orion. 
Its  primary  roles  are  antisubmarine  warfare, 
antisurface  warfare,  and  intelligence,  surveillance, 
and  reconnaissance.  The  P-8A  is  part  of  a  family  of 
systems,  including  the  MQ-4C  Triton  unmanned 
aircraft  system,  which  will  perform  maritime  patrol 
missions  and  support  the  Navy's  maritime 
warfighting  capability.  The  program  plans  to  field 
capabilities  in  three  increments.  We  assessed 
increment  one  and  made  observations  on 
increments  two  and  three. 


Source:  U.S.  Navy. 
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Program  Performance  (fiscal  year  2014  dollars  in  millions) 

As  of 

Latest 

Percent 

06/2004 

08/2013 

change 

Research  and  development  cost 

$7,921.2 

$8,826.6 

11.4 

Procurement  cost 

$24,574.2 

$25,122.3 

2.2 

Total  program  cost 

$32,640.1 

$34,336.0 

5.2 

Program  unit  cost 

$283,827 

$281,442 

-0.8 

Total  quantities 

115 

122 

6.1 

Acquisition  cycle  time  (months) 

160 

164 

2.5 

The  P-8A  entered  production  in  August  2010  with 
mature  technology,  a  stable  design,  and  proven 
production  processes.  The  P-8A  completed  initial 
operational  testing  in  March  2013  and  was  found 
to  be  operationally  suitable  and  effective.  The  test 
report  identified  five  significant  deficiencies  and 
recommended  that  the  P-8A  be  introduced  to  the 
fleet  with  several  operational  restrictions. 
According  to  the  program  office,  it  has  plans  in 
place  to  correct  or  otherwise  resolve  these 
deficiencies.  The  program's  full  rate  production 
decision  was  delayed  from  July  2013  until 
January  2014  in  part  to  enable  additional 
assessment  of  deficiency  corrections  from 
developmental  test  and  evaluation.  To  maintain 
production  continuity,  and  utilize  available 
funding,  the  program  awarded  a  fourth  low-rate 
production  contract.  The  first  P-8A  operational 
deployment  began  in  December  2013. 
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P-8A  Program 

Technology  Maturity 

The  P-8A  entered  production  in  August  2010  with  a 
fully  mature  and  sole  critical  technology,  the  hydro¬ 
carbon  sensor.  The  sonobuoy  launcher,  which  at 
one  time  was  also  considered  a  critical  technology, 
has  experienced  issues  in  testing  and  fixes  are  still 
being  made  to  improve  its  performance  and  reduce 
operator  errors. 

Design  Maturity 

Overall,  the  design  of  the  P-8A  is  stable,  but  the 
number  of  design  drawings  continues  to  grow  due 
to  deficiencies  discovered  in  testing.  The  number  of 
releasable  drawings  increased  by  16  percent  since 
our  last  assessment  and  95  percent  of  the  total 
drawings  have  been  released. 

Production  Maturity 

The  manufacturing  processes  for  P-8A  are  mature 
and  the  program  has  improved  the  acceptance 
process  after  minor  anomalies  and  maintenance 
issues  were  discovered  with  each  of  the  first  five 
production  aircraft.  According  to  program  officials, 
Boeing  has  implemented  additional  pre-acceptance 
flights,  which  have  reduced  the  number  of  issues 
discovered  by  the  government  during  the  formal 
acceptance  process.  More  recently,  the  Defense 
Contract  Management  Agency  reported  the  late 
delivery  of  spare  parts  could  affect  aircraft  deliveries 
by  reducing  the  program's  ability  to  swap  out 
defective  parts  when  needed.  The  program  office 
stated  that  it  has  addressed  this  issue  and  aircraft 
deliveries  had  not  been  impacted  to  date. 

Other  Program  Issues 

The  P-8A  program  completed  initial  operational  test 
and  evaluation  in  March  2013.  The  final  test  report 
found  the  aircraft  to  be  operationally  effective  and 
suitable.  The  report  identified  five  significant 
deficiencies  that  are  typically  corrected  before 
operational  deployment  and  the  operational  test 
staff  recommended  that  the  P-8A  enter  the  fleet  with 
several  operational  restrictions.  According  to  the 
program  office,  it  has  plans  in  place  to  correct  or 
otherwise  resolve  these  deficiencies.  For  example, 
to  maintain  main  fuel  tank  temperatures  within 
required  operating  limits  during  hot  weather  ground 
operations,  the  program  is  re-qualifying  key 
components  to  higher  operating  limits  and 
developing  new  planning  tools  and  fueling 


procedures.  The  first  operational  deployment  of  P- 
8A  commenced  in  December  2013.  Follow-on 
operational  testing,  which  includes  verifying  test 
fixes  discovered  previously  in  developmental 
testing,  began  in  October  2013. 

The  Navy  delayed  the  program's  full  rate  production 
decision  from  July  2013  until  January  2014  in  order 
to  allow  enough  time  for  the  program  to  assess 
deficiency  corrections.  To  maintain  production  line 
continuity,  and  utilize  available  program  funding,  the 
program  awarded  a  fourth  low-rate  initial  production 
contract  for  13  aircraft. 

During  increments  2  and  3,  the  program  plans  to 
integrate  additional  capabilities  into  the  P-8A. 
Increment  2,  which  is  being  managed  as  a  series  of 
engineering  changes  rather  than  a  separate 
development  effort,  includes  various  improvements 
to  the  system's  antisubmarine  warfare  and  other 
capabilities.  The  program  eventually  plans  to  retrofit 
previously  manufactured  aircraft  with  the  planned 
Increment  2  capabilities.  The  Navy  plans  to  conduct 
a  separate  system  development  phase  for 
increment  3.  The  improvements  include  changes  to 
the  P-8A's  system  architecture  to  allow  for  rapid 
integration  of  new  applications  and  services.  The 
program  plans  to  conduct  full  and  open  competition 
for  its  increment  3  contracts  and  to  purchase  the 
data  rights  for  the  new,  more  open  architecture. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  stated  that  following  an  extensive 
operational  test  period,  the  Navy's  operational  test 
authority  reported  that  the  P-8A  aircraft  is 
operationally  effective  and  suitable  and 
recommended  it  for  fleet  introduction.  The  program 
office  added  that  this  conclusion  is  consistent  with 
the  fleet's  experience  with  the  aircraft  during 
extensive  aircraft  transition  and  pre-deployment 
training  cycles.  The  first  such  cycle  began  with  the 
delivery  of  the  first  P-8A  aircraft  to  an  operational 
squadron  in  May  2012,  included  the  attainment  of 
initial  operational  capability  in  November  2013,  and 
culminated  with  the  departure  of  the  first  fully 
trained  squadron  for  operational  deployment  later 
that  same  month.  In  December  2013,  the  thirteenth 
of  37  low-rate  initial  production  aircraft  was 
delivered  to  fleet  squadrons,  with  all  deliveries  on  or 
ahead  of  schedule.  The  program  office  also 
provided  technical  comments,  which  were 
incorporated  as  appropriate. 
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Paladin  Integrated  Management  (PIM) 


The  Army's  Paladin  Integrated  Management  (PIM) 
system  consists  of  two  individual  platforms,  a  self- 
propelled  howitzer  (SPH)  and  a  tracked  ammunition 
carrier  that  provides  operational  support.  The  SPH 
is  a  tracked,  aluminum  armored  vehicle  with  a  155 
millimeter  cannon.  The  PIM  will  provide  improved 
sustainability  over  the  current  Paladin  M109A6 
howitzer  fleet  through  the  incorporation  of  a  newly 
designed  hull;  modified  M2  Bradley  infantry  fighting 
vehicle  powertrain,  suspension  system,  and  track; 
and  a  modernized  electrical  system. 


Source:  U.S.  Army. 
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Program  Performance  (fiscal  year  2014  dollars  in  millions) 


Research  and  development  cost 

As  of 
02/2012 

$1,060.1 

Latest 

10/2013 

$1,101.7 

Percent 

change 

3.9 

Procurement  cost 

$5,973.4 

$5,886.3 

-1.5 

Total  program  cost 

$7,033.5 

$6,987.9 

-0.6 

Program  unit  cost 

$12,085 

$12,523 

3.6 

Total  quantities 

582 

558 

-4.1 

Acquisition  cycle  time  (months) 

118 

118 

0.0 

The  program  was  approved  for  low-rate  initial 
production  in  October  2013  with  the  first 
production  delivery  scheduled  for  March  2015. 
Program  officials  stated  that  a  lack  of  available 
staff,  staff  furloughs,  loss  of  workforce  overtime, 
and  a  reduction  in  travel  funding  due  to 
sequestration  caused  the  production  decision 
date  to  slip  from  June  2013  to  October  2013.  The 
program  was  approved  for  production  with  its  two 
critical  technologies  fully  mature  and  a  stable 
design.  The  program,  as  originally  structured,  was 
not  a  major  defense  acquisition  program,  but  was 
elevated  to  major  defense  acquisition  program 
status  due  to  rising  cost  estimates.  The  program 
has  also  experienced  schedule  slippage  due  to 
delays  in  the  start  of  developmental  testing  and 
changes  to  force  protection  and  survivability 
requirements  that  drove  the  addition  of  new  armor 
kits  and  a  new  ballistic  hull  and  turret. 


Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  a  realistic 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control  O 

•  Demonstrate  critical  processes  on  a  pilot  production  line  o 

•  Test  a  production-representative  prototype  o 


9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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PIM  Program 

Technology  Maturity 

The  program  office  identified  two  technologies 
critical  to  the  PIM  program;  power  pack  integration 
and  the  ceramic  bearing  of  the  generator  assembly. 
Both  technologies  have  been  assessed  as  fully 
mature.  The  program  office  identified  these 
technologies  as  critical  based  on  concerns  about 
their  performance  at  high  temperatures.  While 
neither  technology  is  new  or  novel,  failure  of  either 
would  represent  major  program  risk.  In  addition, 
increases  in  vehicle  weight  due  to  the  addition  of 
material  to  improve  survivability,  minimal  power 
pack  space,  and  other  integration  issues  may 
degrade  the  ability  to  meet  automotive  performance 
requirements. 

Design  Maturity 

According  to  the  program  office,  the  design  is 
currently  stable  with  all  of  the  expected  drawings 
released;  however,  the  program  faces  a  number  of 
design  challenges  as  it  moves  forward.  The  current 
contractor  for  the  engine  and  transmission  may 
cease  production  due  to  lack  of  orders.  If  this 
occurs,  production  with  another  vendor  may  be 
necessary  and  could  result  in  a  redesign  of  the 
engine  and  engine  compartment.  Production  with 
another  vendor  has  the  potential  for  cost  growth 
between  $32  to  $100  million  and  a  significant 
schedule  delay.  Additionally,  after  the  critical  design 
review,  the  program  manager  identified  a  number  of 
corrective  actions,  producibility,  and  obsolescence 
engineering  changes  to  the  PIM  design  that  are 
scheduled  to  be  implemented  between  low-rate 
initial  production  and  initial  operational  test  and 
evaluation.  If  significant  issues  are  uncovered 
during  the  testing  of  these  engineering  changes, 
then  there  is  potential  for  cost  growth  and  delays  in 
production. 

Production  Maturity 

The  program  was  approved  for  low-rate  initial 
production  in  October  2013  with  the  first  production 
vehicle  scheduled  for  delivery  in  March  2015.  As  of 
August  2013,  the  program  office  indicated  that  a 
number  of  manufacturing  processes  and  systems 
are  below  the  maturity  levels  required  by  best 
practices.  The  program  reported  that  the  capability 
to  produce  the  vehicles  in  a  pilot-line  environment 
will  be  not  be  demonstrated  until  after  production 
begins.  Our  best  practices  work  has  shown  that 


programs  risk  missing  cost  and  schedule  targets  if 
their  manufacturing  processes  are  not 
demonstrated  and  in  control  before  production 
begins.  Program  officials  have  indicated  that  there 
is  a  minimal  risk  that  the  PIM  final  assembly  facility 
may  not  be  ready  in  time  to  support  initial  PIM 
shipments. 

Other  Program  Issues 

The  program's  current  plans  are  schedule  driven, 
with  limited  time  for  correction  of  deficiencies 
identified  in  developmental  testing  and  little 
flexibility  with  individual  component  test  schedules. 
The  program  is  waiting  for  low-rate  initial  production 
vehicles  before  verifying  most  corrective  actions. 
Additionally,  program  officials  stated  that  the  PIM 
and  the  Bradley  infantry  fighting  vehicle  programs 
currently  share  a  common  path  for  the  suspension 
and  powertrain,  but  if  the  Bradley  program  diverges 
from  this  path  there  could  be  cost  and  schedule 
impacts  for  the  PIM  program. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Army  stated  that  the  PIM  program  is  on  track  for  its 
full-rate  decision  scheduled  for  January  2017. 
Additionally,  technical  comments  were  provided, 
which  were  incorporated  as  appropriate. 
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Presidential  Helicopter  Fleet  Replacement  Program  (VXX) 


The  Navy's  VXX  program  is  to  provide  a 
replacement  helicopter  fleet  that  will  be  used  to 
transport  the  President,  Vice  President,  heads  of 
state,  and  others.  As  a  successor  to  the  terminated 
VH-71  program,  the  VXX  fleet  of  21  new  helicopters 
will  replace  the  current  fleet  of  19  VH-3D  and  VH- 
60N  legacy  helicopters  and  two  training  aircraft  (to 
be  delivered  in  2014).  The  program  plans  to  enter 
system  development  in  March  2014.  Until  the  VXX 
helicopters  are  available,  the  Navy  is  extending  the 
availability  of  the  legacy  aircraft. 


Source:  U.S.  Navy. 
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Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Latest  Percent 

As  of 

11/2013 

change 

Research  and  development  cost 

NA 

$2,532.0 

NA 

Procurement  cost 

NA 

$0.0 

NA 

Total  program  cost 

NA 

$2,523.0 

NA 

Program  unit  cost 

NA 

TBD 

NA 

Total  quantities 

NA 

23 

NA 

Acquisition  cycle  time  (months) 

Table  includes  costs  through  fiscal  year  2018. 

NA 

77 

NA 

► 


The  Navy  made  progress  in  establishing  a  limited 
development  effort  that  reflects  trade-offs  made 
among  cost,  schedule,  risk,  and  performance. 
Since  April  2012,  the  program  completed  its 
analysis  of  alternatives,  in  which  affordability  and 
cost  control  are  key  factors,  finalized  the 
requirements  and  acquisition  strategy,  and  was 
granted  a  waiver  to  the  competitive  prototyping 
requirement.  Program  affordability  and  timeliness 
are  achieved  by  maintaining  a  candidate 
platform's  current  airworthiness  certification, 
leveraging  mature  technologies,  and  through 
Government  definition  of  the  mission  systems 
architecture  and  related  avionics.  To  minimize 
risk,  a  sub-system  preliminary  technical  readiness 
assessment  was  conducted  in  February  2013. 
However,  a  system  level  preliminary  design 
review  has  been  deferred  until  after  the  March 
2014  start  of  system  development. 
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VXX  Program 

Technology  Maturity 

The  program  reports  the  key  technologies  to  be 
provided  the  government  for  integration  into  the 
VXX  platform  either  already  exist  or  are  in 
development,  some  as  legacy  fleet  aircraft 
upgrades.  To  minimize  technical  risk,  the  program 
completed  a  preliminary  technical  readiness 
assessment  in  February  2013;  it  was  limited  to  the 
mission  communication  system  (the  only  new 
technology  envisioned  for  the  VXX  platform).  As  a 
result  of  this  assessment,  it  was  deemed  the  VXX 
mission  communication  system  does  not  include 
any  components  that  would  be  considered 
immature  and  classified  as  a  critical  technology 
element.  Although  the  program  has  conducted  a 
sub-system  preliminary  design  review,  it  does  not 
currently  plan  to  conduct  a  preliminary  design 
review  until  after  the  program  enters  system 
development.  The  statutory  requirement  for  a 
preliminary  design  review  has  been  waived,  and  will 
be  conducted  after  the  start  of  system  development. 
To  be  consistent  with  the  program's  approved 
acquisition  strategy,  the  language  in  the  request  for 
proposals  encourages  the  use  of  mature  off-the- 
shelf  technologies  that  should  only  require  standard 
engineering  development  practice  to  integrate  and 
achieve  installed  systems  performance. 

Design  Maturity 

The  VXX  acquisition  strategy  is  based  on  the 
integration  of  mature  subsystems  into  an  existing  air 
vehicle.  While  minor  changes  to  the  platform,  to 
accommodate  integration  of  subsystems,  are 
inevitable,  change  to  major  components  such  as 
drive  train,  rotors,  engines  and  basic  structure  is 
highly  discouraged.  Any  design  elements  that 
contain  immature  technology  or  that  might  be 
deemed  a  critical  technology  element  are 
discouraged.  The  request  for  proposals  encouraged 
offerors  to  remain  within  the  existing  airworthiness 
qualification  of  the  baseline  aircraft  from  which  the 
VXX  would  be  derived.  The  government  does  not 
intend  to  impose  bottom-up  airworthiness  criteria 
that  would  drive  redesign,  but  plans  to  leverage  the 
current  certification  basis  with  the  cognizant  civil  or 
U.S.  military  airworthiness  certification  authority  to 
achieve  a  Naval  Air  Systems  Command  flight 
clearance. 


Production  Maturity 

This  program  will  leverage  "green  air  vehicle" 
production  by  randomly  selecting  aircraft  off  the 
existing  production  line.  This  will  save  the 
government  time  and  money  by  avoiding 
infrastructure  costs  associated  with  constructing  a 
unique  production  facility  and  allow  the  program  to 
meet  its  security  production  and  supply 
requirements. 

Other  Program  Issues 

The  program  intends  to  award  a  single  fixed-price 
type  contract  for  the  system  development  phase 
that  will  include  options  for  low-rate  initial 
production,  full  rate  production,  and  associated 
logistics  elements.  The  total  aircraft  quantity 
includes  six  aircraft  to  support  system  development, 
17  production  aircraft  in  three  lots  (six  aircraft  each 
in  lot  1  and  2,  and  5  in  lot  3).  The  total  period  of 
performance  will  be  approximately  8  years. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Common  Name:  RQ-4A/B  UAS  Global  Hawk 


RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System  (RQ-4A/B  Global  Hawk) 


The  Air  Force's  Global  Hawk  is  a  high-altitude,  long- 
endurance  unmanned  aircraft  that  provides 
intelligence,  surveillance,  and  reconnaissance 
capabilities.  After  a  successful  technology 
demonstration,  the  system  entered  development 
and  limited  production  in  March  2001.  The  early 
RQ-4A,  similar  to  the  original  demonstrators,  was 
retired  in  2011,  leaving  a  fleet  of  the  larger  and  more 
capable  RQ-4Bs,  produced  in  three 
configurations — Block  20,  30,  and  40. 


Source:  Northrop  Grumman. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Northrop  Grumman 

As  of 

Latest 

Percent 

Program  office:  Wright-Patterson  AFB, 

03/2001 

08/2013 

change 

OH 

Research  and  development  cost 

$1,095.6 

$4,028.2 

267.7 

Funding  needed  to  complete: 

Procurement  cost 

$4,542.3 

$5,708.1 

25.7 

R&D:  $318.6  million 

Total  program  cost 

$5,671.1 

$9,874.5 

74.1 

Procurement:  $170.1  million 

Program  unit  cost 

$90,017 

$219,432 

143.8 

Total  funding:  $488.7  million 

Total  quantities 

63 

45 

-28.6 

Procurement  quantity:  0 

Acquisition  cycle  time  (months) 

55 

TBD 

TBD 

The  Global  Hawk  program  has  mature 
technologies,  a  stable  design,  and  mature 
manufacturing  processes  that  are  in  statistical 
control.  Congress  directed  the  program  to 
purchase  the  final  three  Block  30  aircraft  and  the 
program  office  awarded  an  advanced 
procurement  contract  in  September  2013.  All  45 
Global  Hawk  aircraft  have  been  purchased 
without  a  full-rate  production  decision.  Congress 
directed  the  program  to  maintain  the  operational 
capability  of  Block  30  aircraft  through  December 
2014.  The  Air  Force  will  determine  the  final  force 
structure  following  fiscal  year  2015  and  2016 
budget  decisions.  Block  40  aircraft  operational 
testing  has  been  delayed  6  months  and  delivery 
of  a  fully  operational  Block  40  capability  has  not 
been  determined.  The  program  fielded  two  Block 
40  aircraft  in  September  2013,  providing  limited 
capability  to  the  warfighter. 
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Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  in  a  relevant 
environment 


Demonstrate  all  critical  technologies  in  a  realistic 
environment 


Complete  preliminary  design  review 


Demonstrate  critical  processes  are  in  control 
Demonstrate  critical  processes  on  a  pilot  production  line 
Test  a  production-representative  prototype 


0  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 
Not  applicable 


Common  Name:  RQ-4A/B  UAS  Global  Hawk 


RQ-4A/B  UAS  Global  Hawk  Program 

Technology,  Design,  and  Production  Maturity 

The  critical  technologies  for  the  RQ-4B  are  mature, 
the  basic  airframe  design  is  stable,  and  its 
manufacturing  processes  are  mature  and  in 
statistical  control.  The  RQ-4B  aircraft  consist  of 
three  configurations.  Block  20  aircraft  are  equipped 
with  an  enhanced  imagery  intelligence  payload, 
Block  30  aircraft  have  both  imagery  and  signals 
intelligence  payloads,  and  Block  40  aircraft  have  an 
advanced  radar  surveillance  capability.  The 
program  is  concurrently  testing  and  fielding  the 
multiple  platform  radar,  the  key  capability  for  Block 
40  aircraft.  Due  to  software  delays  associated  with 
another  government  system,  Block  40  operational 
testing  has  slipped  an  additional  9  months,  from 
January  2014  to  October  2014,  almost  4  years  from 
the  original  estimate.  Despite  this,  two  Block  40 
aircraft  with  the  advanced  radar  capability  were 
fielded  in  September  2013,  providing  a  limited 
capability  to  the  warfighter.  Program  officials  stated 
that  there  is  currently  no  formal  date  for  the  program 
to  deliver  a  full  capability  Block  40  aircraft  for 
operational  use. 

Other  Program  Issues 

As  part  of  its  fiscal  year  201 3  budget  request,  DOD 
proposed  terminating  the  Block  30  system  in  an 
effort  to  reduce  program  cost.  However,  in  January 
2013,  Congress  directed  the  Global  Hawk  program 
to  purchase  its  remaining  three  planned  Block  30 
aircraft  and  mandated  that  the  Air  Force  maintain 
the  operational  capability  of  Block  30  aircraft  until 
December  31,  2014.  The  program  reported  that 
operating  Block  30  aircraft  beyond  2014  would 
create  operational  challenges  and  would  require  a 
substantial  investment.  According  to  officials,  the  Air 
Force  will  make  a  decision  about  the  program  force 
structure  following  future  budget  decisions.  If  the  Air 
Force  decides  to  retire  Block  30  aircraft,  almost  half 
of  all  aircraft  procured  will  be  affected.  In 
September  2013,  the  Air  Force  awarded  an 
advanced  procurement  contract  for  the  remaining 
three  Block  30  aircraft  and  intends  to  award  a  firm 
fixed-price  production  contract  in  July  2014. 

The  Global  Hawk  program  continues  to  operate 
without  an  established  cost  and  schedule  baseline, 
as  its  prior  baseline  was  rescinded  due  to  the 
program's  Nunn-McCurdy  breach  in  April  2011 .  The 
program  had  originally  planned  to  re-establish  its 


baseline  in  June  2011 ,  but  that  review  was 
superseded  by  the  Air  Force's  proposed  termination 
of  Block  30.  Program  officials  stated  they  are 
working  with  the  Office  of  the  Secretary  of  Defense 
and  the  Air  Force  to  establish  a  new  baseline. 

Production  continues  for  Block  30  and  Block  40 
aircraft,  despite  the  lack  of  an  official  full-rate 
production  review.  All  45  Global  Hawk  aircraft  have 
been  procured  through  low-rate  initial  production. 
The  program  does  not  plan  to  hold  a  production 
decision  and,  according  to  officials,  continues  to 
sustain  operational  aircraft. 

The  program  had  planned  to  begin  development  of 
a  restructured  ground  control  station  and 
communications  system  to  achieve  a  common 
architecture  for  both  the  Global  Hawk  and  the 
Navy's  Triton  unmanned  aircraft  systems,  has  a 
similar  design  to  Global  Hawk.  However,  according 
to  program  officials  the  program  could  no  longer 
afford  the  development  effort  due  to  fiscal 
constraints.  Program  officials  noted  that  they  have 
chosen  to  continue  to  maintain  the  current  system 
and  address  obsolescence  issues  on  an  as-needed 
basis. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  Air 
Force  officials  noted  that  the  Air  Force  Operational 
Test  and  Evaluation  Center  completed  the  Block  40 
operational  utility  evaluation  in  April  2013  in  support 
of  a  Block  40  early  operational  capability.  The  two 
Block  40  aircraft  that  fielded  in  September  2013 
were  part  of  the  early  operational  capability  which 
provided  the  military  with  additional  ground  moving 
target  support  ahead  of  schedule.  The  Air  Force 
also  provided  technical  comments,  which  were 
incorporated  as  appropriate. 
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Common  Name:  SSC 


Ship  to  Shore  Connector  Amphibious  Craft  (SSC) 

The  Navy's  SSC  is  an  air-cushioned  landing  craft 
intended  to  transport  personnel,  weapon  systems, 
equipment,  and  cargo  from  amphibious  vessels  to 
shore.  SSC  is  the  replacement  for  the  Landing 
Craft,  Air  Cushion,  which  is  approaching  the  end  of 
its  service  life.  The  SSC  is  designed  to  deploy  in 
Navy  well  deck  amphibious  ships,  such  as  the  LPD 
17  class,  and  for  use  in  assault  and  nonassault 
operations.  The  program  entered  system 
development  in  July  2012  and  is  scheduled  for  a 
critical  design  review  in  March  2014. 


Source:  U.S.  Navy. 


\ 

A 


Concept  System  development 


Production 


A 

A 

A 

A 

A 

A 

A 

A 

Program 

Development 

GAO 

Design 

Low-rate 

Start 

Full-rate 

Initial 

start 

start 

review 

review 

decision 

operational  test 

decision 

capability 

(5/09) 

(7/12) 

(1/14) 

(3/14) 

(11/14) 

(1/17) 

(9/18) 

(8/20) 

► 


Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Textron  Inc. 

As  of 

Latest 

Percent 

Program  office:  Washington,  DC 

07/2012 

09/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$585.4 

$571.2 

-2.4 

R&D:  $166.8  million 

Procurement  cost 

$3,552.6 

$3,462.9 

-2.5 

Procurement:  $3,462.9  million 

Total  program  cost 

$4,157.6 

$4,053.3 

-2.5 

Total  funding:  $3,648.9  million 

Program  unit  cost 

$56,953 

$55,524 

-2.5 

Procurement  quantity:  71 

Total  quantities 

73 

73 

0.0 

Acquisition  cycle  time  (months) 

135 

135 

0.0 

The  SSC  program  entered  system  development 
in  July  2012  with  its  one  critical  technology — the 
fire  suppressant  system — mature.  Two  other 
technologies  designated  as  technology  watch 
items  leverage  existing  mature  technologies,  but 
will  need  to  be  modified  for  use  on  SSC.  The 
program  has  completed  approximately  5  percent 
of  expected  design  drawings  seven  months  prior 
to  critical  design  review  scheduled  for  March 
2014.  Fabrication  for  the  first  two  craft  is 
scheduled  to  start  three  months  after  the  critical 
design  review.  The  acquisition  strategy  identified 
the  plan  to  construct  the  first  two  craft 
concurrently  as  a  risk  due  to  the  potential  for 
design  changes. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  SSC 


SSC  Program 

Technology  Maturity 

The  SSC  program's  one  identified  critical 
technology,  the  fire  suppression  system,  entered 
system  development  as  mature.  According  to 
program  officials,  the  prime  contractor  is  in  the 
process  of  selecting  a  subcontractor  to  develop  this 
system.  According  to  the  Navy,  the  system  is 
expected  to  be  lighter  and  more  environmentally 
friendly  than  the  system  for  the  legacy  craft. 

DOD  identified  two  other  technologies — the  gas 
turbine  engine  and  the  command,  control, 
communications,  computer  and  navigation  (C4N) 
system — as  critical  technology  watch-list  items 
because  of  the  possibility  new  technologies  would 
need  to  be  developed  or  modified  for  SSC. 
According  to  program  officials,  the  gas  turbine 
engine  is  mature,  but  needs  modifications  to  convert 
it  from  aircraft  to  amphibious  craft  usage.  In 
addition,  program  officials  said  that  the  prime 
contractor  has  selected  a  subcontractor  to  develop 
the  C4N  system  and  believes  that  the  technology  is 
low  risk  since  it  will  leverage  existing  code  from  the 
legacy  craft.  Officials  expect  that  the  system  will 
provide  new  operational  features,  including  a 
pilot/co-pilot  configuration  that  allows  each  operator 
to  bring  up  the  same  display  at  separate  consoles. 

Design  Maturity 

As  of  August  2013,  the  contractor  has  released  42 
of  the  856  engineering  drawings  expected — almost 
5  percent.  Best  practices  suggest  that  completion  of 
90  percent  of  engineering  drawing  provides 
evidence  of  design  stability.  The  program  plans  to 
demonstrate  the  stability  of  the  SSC  design  at  the 
program's  production  readiness  review,  currently 
planned  for  May  2014 — 2  months  after  the 
program's  scheduled  design  review. 

Fabrication  of  the  first  two  craft  are  scheduled  to 
begin  in  June  2014.  The  program  office  does  not 
plan  to  assess  critical  manufacturing  processes  by 
the  start  of  production.  Program  officials  stated  that 
they  plan  to  have  these  processes  partially 
demonstrated  on  a  pilot  production  line  before 
production  start,  noting  that  the  program  has 
pursued  low  risk  technologies  and  a  low  risk  design 
effort. 


Production  Maturity 

The  Navy  plans  to  begin  construction  of  the  lead 
and  follow-on  craft  concurrently — a  strategy  that 
has  been  identified  as  potentially  risky  due  to  the 
possibility  that  design  changes  identified  in  the  first 
test  and  training  craft  would  have  to  be  incorporated 
in  the  second  craft  during  construction.  Despite  this 
risk,  the  SSC  program  exercised  an  option  to 
construct  the  second  craft  in  December  2012 — 3 
months  earlier  than  planned.  Program  officials 
noted  that  the  option  was  exercised  earlier  to  take 
advantage  of  better  pricing  for  certain  long-lead 
construction  materials.  Overall,  the  program  plans 
to  exercise  options  for  a  total  of  six  craft  prior  to 
delivery  of  the  test  and  training  craft  in  February 
2017. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  the  Ship  to  Shore 
Connector  is  a  technically  mature,  low  risk  program 
which  is  proceeding  toward  demonstration  of  design 
maturity.  A  detail  design  and  construction  contract 
was  awarded  in  July  2012.  The  shipbuilder  is 
conducting  preliminary  and  critical  design  reviews  of 
all  major  craft  subsystems.  The  first  craft's  start  of 
fabrication  is  planned  for  the  third  quarter  of  fiscal 
year  2014.  The  program  office  also  provided 
technical  comments  which  were  incorporated  where 
deemed  appropriate. 
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Common  Name:  SDB  II 


Small  Diameter  Bomb  Increment  II  (SDB  II) 


The  Air  Force's  Small  Diameter  Bomb  Increment  II 
(SDB  II)  is  designed  to  provide  attack  capability 
against  mobile  targets  in  adverse  weather  from 
standoff  range.  It  combines  radar,  infrared,  and 
semiactive  laser  sensors  in  a  tri-mode  seeker  to 
acquire,  track,  and  engage  targets.  It  uses  airborne 
and  ground  data  links  to  update  target  locations  as 
well  as  GPS  and  an  inertial  navigation  system  to 
ensure  accuracy.  SDB  II  will  be  integrated  with  F- 
15E,  Joint  Strike  Fighter  (JSF),  F/A-18E/F,  F-16C, 
and  F-22A,  as  funding  becomes  available. 


Source:  ©  2009  Raytheon  Company. 
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Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  Raytheon 

As  of 

Latest 

Percent 

Program  office:  Eglin  AFB,  FL 

10/2010 

08/2013 

change 

Funding  needed  to  complete: 

Research  and  development  cost 

$1,729.7 

$1,649.5 

-4.6 

R&D:  $612.3  million 

Procurement  cost 

$3,215.2 

$2,177.3 

-32.3 

Procurement:  $2,175.3  million 

Total  program  cost 

$4,944.9 

$3,826.7 

-22.6 

Total  funding:  $2,787.6  million 

Program  unit  cost 

$0,288 

$0,223 

-22.6 

Procurement  quantity:  17,000 

Total  quantities 

17,163 

17,163 

0.0 

Acquisition  cycle  time  (months) 

72 

79 

9.7 

The  SDB  II  Program  expects  all  of  its  critical 
technologies  to  reach  full  maturity  by  the  time  it 
enters  low-rate  production,  which  is  planned  to 
occur  no  earlier  than  August  2014 — a  7-month 
delay  from  earlier  estimates.  The  flight  test 
program  has  experienced  two  failures,  prompting 
a  6  month  halt  to  flight  testing  in  mid-2013.  Since 
resuming  flight  test  activities  in  October  2013,  the 
program  has  conducted  three  successful  flight 
tests.  Although  the  design  is  stable,  future  design 
changes  may  be  needed  to  address  problems 
found  in  previous  flight  test  failures.  The  flight  test 
schedule  is  optimistic,  requiring  seven  successful 
flight  tests  in  5  months.  If  additional  flight  test 
failures  occur  it  could  result  in  additional  delays 
for  the  low-rate  production  decision. 


Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant  _ 

environment  ® 

•  Demonstrate  all  critical  technologies  in  a  realistic  p, 

environment 

•  Complete  preliminary  design  review  0 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings  0 

•  Test  a  system-level  integrated  prototype  % 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

Q  Knowledge  attained  ■  ■  ■  ■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  SDB  II 


SDB  II  Program 

Technology  Maturity 

SDB  ll's  four  critical  technologies — guidance  and 
control,  multi-mode  seeker,  net  ready  data  link,  and 
payload — are  scheduled  to  reach  full  maturity  when 
the  program  has  its  low-rate  production  decision  in 
August  2014,  4  years  after  the  program  entered 
system  development.  According  to  program 
officials,  critical  technologies  have  not  directly 
contributed  to  any  flight  test  failures. 

Design  Maturity 

The  number  of  design  drawings  is  stable  and  the 
program  has  completed  five  successful  guided  flight 
tests.  However,  the  flight  test  program  experienced 
two  failures  that  resulted  in  a  6-month  halt  in  all 
guided  test  vehicle  events  until  the  program 
determined  a  root  cause  for  the  failures.  The  two 
failures  were  the  result  of  the  dome  cover  not 
deploying,  preventing  the  seeker  from  acquiring  the 
target,  and  a  navigation  error.  Guided  flight  test 
activities  resumed  in  October  2013  and,  since  then, 
the  program  has  recorded  three  successful  flight 
tests.  Program  officials  stated  that  all  of  the  failed 
tests  have  to  be  retested  at  a  future  date.  Moreover, 
prior  to  the  low-rate  production  decision,  the 
program's  test  plan  requiresll  successful  flight 
tests,  including  two  live  fire  events. 

Production  Maturity 

The  program's  low-rate  production  decision  is 
expected  no  earlier  than  August  2014,  a  7-month 
slip  due  to  a  delay  in  the  program's  system 
verification  review,  a  prerequisite  for  this  decision. 
The  verification  review  will  evaluate  all  of  the 
program's  flight  tests  and  the  system  qualification 
testing.  Eleven  of  the  12  subsystems  have 
successfully  passed  qualification  testing.  However, 
all  up  round  environmental  qualification  testing  has 
been  on  hold  since  September  2013  due  to 
subsystem  issues. 

According  to  program  officials,  the  seeker 
subsystem  is  the  final  subsystem  to  complete 
qualification.  The  subsystem  qualification  was 
halted  for  issues  related  to  a  leak  in  the  warhead 
case  and  seeker  encoders  that  failed  during 
vibration  testing.  A  correction  has  been  identified  for 
the  encoders  and  will  be  incorporated  before  testing 
resumes.  The  all  up  round  qualification  program  has 
been  placed  on  hold  until  the  leak  in  the  warhead 


case  is  resolved.  The  program  has  yet  to  identify  a 
correction  for  the  leak  in  the  warhead.  These  delays 
have  absorbed  the  entire  1 -month  qualification 
testing  margin  and  contributed  to  the  delayed 
system  verification  review,  now  scheduled  for  May 
2014.  According  to  program  officials,  the  contractor 
will  be  adding  additional  vehicles  to  the  all  up  round 
qualification  program  to  lessen  the  effect  that  the 
pause  has  on  the  overall  program  schedule. 

Other  Program  Issues 

The  program's  test  schedule  prior  to  the  low-rate 
production  decision  is  optimistic.  In  addition  to  a  5- 
month  delay  in  2012,  the  test  program  recently 
ended  a  6-month  delay  due  to  two  flight  test  failures. 
According  to  program  officials,  the  test  program 
must  have  six  successful  flight  tests  prior  to  the  May 
2014  verification  review.  Additional  flight  test 
failures  would  likely  further  delay  the  low-rate 
production  decision.  Program  officials  also  noted 
that  future  sequestration  funding  cuts  may  result  in 
reduced  production  quantities  or  the  inability  to 
exercise  contract  options.  This  could  force  a 
renegotiation  with  the  contractor  for  the  first  five 
production  lots  and  the  program  could  lose  money 
that  it  saved  through  competition. 

Program  Office  Comments 

The  Air  Force  concurred  with  this  assessment.  The 
Air  Force  also  provided  technical  comments,  which 
were  incorporated  as  appropriate. 
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Common  Name:  Space  Fence 


Space  Fence 

The  Air  Force's  Space  Fence  program  is  developing 
a  system  of  large  ground-based  radars  that  will 
replace  the  Air  Force  Space  Surveillance  System, 
which  became  operational  in  1961  and  was  recently 
shut  down.  Space  Fence  will  use  higher  radio 
frequencies  to  detect  and  track  more  and  smaller 
earth-orbiting  objects  than  is  currently  possible.  The 
Air  Force  currently  plans  to  award  a  contract  for 
construction  of  one  radar  with  an  option  for  a 
second. 


Source:  U.S.  Air  Force. 


Concept 


Program 

start 

(3/09) 


System  development 


Production 


GAO  Development 
review  start 

(1/14)  (5/14) 


Program  Essentials 

Prime  contractor:  TBD 
Program  office:  Hanscom  AFB,  MA 
Funding  needed  to  complete: 

R&D:  $958.2  million 
Procurement:  $103.2  million 
Total  funding:  $1,061.4  million 
Procurement  quantity:  1 


▲  ▲ 

Final  development/  Design 

production  contract  review 

(5/14)  (2/15) 

▲ 

Start 

operational  test 
(6/18) 

▲ 

Initial 

capability 

(11/18) 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Latest  Percent 

As  of  10/2013  change 

Research  and  development  cost 

NA 

$1,565.2 

NA 

Procurement  cost 

NA 

$103.3 

NA 

Total  program  cost 

NA 

$1,668.5 

NA 

Program  unit  cost 

NA 

$1,668,463 

NA 

Total  quantities 

NA 

1 

NA 

Acquisition  cycle  time  (months) 

NA 

116 

NA 

► 


The  Space  Fence  program  has  seven  critical 
technologies,  which  are  expected  to  demonstrate 
full  maturity  during  or  after  the  critical  design 
review.  The  program  delayed  development  start 
and  awarding  of  the  system  development  contract 
to  May  2014,  a  delay  of  almost  2  years.  As  a 
result,  the  program  delayed  initial  operational 
capability  by  a  year  to  November  2018.  DOD 
budget  reductions  also  led  to  the  early  shutdown 
of  the  Air  Force  Space  Surveillance  System, 
which  was  originally  planned  to  be  operational 
through  Space  Fence's  initial  operations.  A  new 
data  processing  system  is  being  developed  by  the 
Air  Force,  separate  from  Space  Fence,  to 
accommodate  the  projected  increase  in  the 
volume  of  data  generated. 
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Attainment  of  Product  Knowledge 


Projected  as  of  January  2014 


Resources  and  requirements  match 


•  Demonstrate  all  critical  technologies  in  a  relevant 
environment 

•  Demonstrate  all  critical  technologies  in  a  realistic 
environment 

•  Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 

^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  Space  Fence 


Space  Fence  Program 

Technology  Maturity 

All  seven  of  the  program's  critical  technologies  are 
currently  nearing  full  maturity  and  are  not  expected 
to  demonstrate  full  maturity  until  the  critical  design 
review,  scheduled  for  9  months  after  development 
start,  or  later.  According  to  program  officials,  the 
critical  technologies  are  not  particularly  challenging 
advances  in  technology.  The  risk  for  the  program  is 
in  deploying  radars  on  the  very  large  scale  needed. 
The  program's  technology  development  phase 
included  competitive  development  of  two  fully 
working  prototypes  by  separate  contractors  to 
reduce  program  risk.  If  a  second  radar  site  is 
deemed  necessary,  the  technologies  used  for  that 
site  may  differ,  depending  on  the  final  program 
design. 

Design  and  Production  Maturity 

The  contract  for  the  final  Space  Fence  system 
development  contract  has  not  yet  been  awarded,  so 
the  program  does  not  have  a  final  system  design 
and  thus  we  did  not  assess  design  or  production 
maturity. 

Other  Program  Issues 

The  Air  Force  plans  to  competitively  award  a  single 
system  development  and  production  contract  in 
May  2014.  This  date  reflects  the  second  significant 
delay  to  contract  award  from  the  original  date  of  July 
2012.  The  first  delay  was  due  to  an  internal  DOD 
program  review  required  before  the  contract 
proposal  could  be  issued.  The  program  office 
attributes  the  additional  delay  in  awarding  the 
contract  and  initiating  development  to  the  strategic 
choices  management  review,  which  looked  at 
affordability  of  future  acquisitions  DOD-wide. 

The  Air  Force  plans  to  award  a  fixed-price  incentive 
contract  for  system  development  activities  for  the 
first  radar  site,  with  a  contract  option  for  the  second 
site.  If  the  option  for  the  second  site  is  exercised,  it 
is  planned  to  be  operational  36  months  after  the 
program  meets  initial  operational  capability. 
However,  funding  for  the  second  site  is  uncertain, 
and  it  potentially  may  not  be  needed  depending  on 
whether  the  first  site  exceeds  its  design  capability. 
Program  officials  stated  that  the  decision  on 
whether  or  not  to  build  a  second  site  will  likely  be 
made  after  the  first  site  is  operational.  In  addition, 


the  program  has  pushed  back  the  anticipated  initial 
operational  capability  date  by  more  than  a  year,  to 
2018. 

The  Air  Force  Space  Surveillance  System,  the 
legacy  system  that  Space  Fence  will  replace,  ended 
operations  in  October  201 3.  The  Air  Force  originally 
intended  to  keep  that  system  functioning  until  Space 
Fence  became  operational.  According  to  the  Air 
Force,  the  decision  to  end  operations  early  was 
made  due  to  funding  constraints  caused  by 
sequestration.  Air  Force  officials  said  they  have 
modified  the  operations  of  other  space  surveillance 
assets  in  a  way  that  will  maintain  a  roughly 
equivalent  level  of  space  surveillance  until  Space 
Fence  is  operational. 

The  Joint  Space  Operations  Center  (JSpOC)  at 
Vandenberg  AFB  is  acquiring  a  new  data 
processing  capability — JSpOC  Mission  System 
(JMS) — to  process  the  increased  volume  of  data 
expected  from  Space  Fence.  JMS  is  being 
developed  through  a  separate  acquisition  program 
and,  according  to  Space  Fence  program  officials,  is 
required  to  process  the  amount  of  data  Space 
Fence  is  expected  to  generate.  Currently,  JMS  is 
scheduled  to  become  operational  by  December 
2016,  enabling  input  and  processing  of  data  from 
Space  Fence  in  time  for  its  initial  operational 
capability. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
program  office  noted  that  the  Office  of  the  Secretary 
of  Defense  chose  to  delay  the  Space  Fence  system 
development  certification  in  light  of  the  strategic 
choices  Management  review  outcomes.  The  Air 
Force  amended  the  Space  Fence  request  for 
proposals  issued  in  November  2013.  This 
amendment  reflects  the  change  to  the  initial 
operational  capability  delivery  date,  specified  as  56 
months  after  contract  award  (first  quarter  fiscal  year 
2019),  as  well  as  a  full  operational  capability  date, 
specified  as  36  months  after  initial  operational 
capability  (first  quarter  fiscal  year  2022).  It  also 
reflects  an  update  to  the  total  projected  budget  to 
more  closely  reflect  the  current  fiscal  reality.  Having 
resumed  source  selection,  award  of  a  single  fixed 
price  incentive  contract  is  expected  in  the  third 
quarter  of  fiscal  year  2014.  The  program  also 
provided  technical  comments,  which  were 
incorporated  where  appropriate. 
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Common  Name:  WIN-T  Increment  2 


Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 


WIN-T  is  the  Army's  high-speed  and  high-capacity 
backbone  communications  network.  It  connects 
units  with  higher  levels  of  command  and  provides 
the  Army's  tactical  portion  of  the  Global  Information 
Grid.  WIN-T  was  restructured  following  a  March 
2007  Nunn-McCurdy  unit-cost  breach  of  the  critical 
threshold,  and  will  be  fielded  in  four  increments.  We 
assessed  the  second  increment,  which  is  expected 
to  provide  the  Army  with  an  initial  networking  on- 
the-move  capability. 


Source:  U.S.  Army. 


ancept  System  development 


Production 


Program/  Design 

development  start  review 

(6/07)  (2/08) 


Low-rate  Initial  operational  Follow-on  Initial  GAO  Follow-on  Full-rate 

decision  testing  operational  test  capability  review  operational  test  decision 

(2/10)  (5/12)  (5/13)  (8/13)  (1/14)  (10/14)  (5/15) 


Program  Essentials 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

Prime  contractor:  General  Dynamics  C4 

As  of 

Latest 

Percent 

Systems,  Inc. 

10/2007 

08/2013 

change 

Program  office:  Aberdeen  Proving 

Research  and  development  cost 

$250.9 

$296.6 

18.2 

Ground,  MD 

Procurement  cost 

$3,649.2 

$4,749.6 

30.2 

Funding  needed  to  complete: 

Total  program  cost 

$3,900.2 

$5,046.2 

29.4 

R&D:  $22.3  million 

Program  unit  cost 

$2,060 

$2,341 

13.6 

Procurement:  $2,285.1  million 

Total  quantities 

1,893 

2,156 

13.9 

Total  funding:  $2,307.3  million 

Acquisition  cycle  time  (months) 

50 

74 

48.0 

Procurement  quantity:  661 

The  Army  was  directed  in  September  2013  to  update  the  cost  position  and  program  baseline.  These 

changes  are  not  reflected  in  the  latest  column  above 

WIN-T  Increment  2  entered  production  in 
February  2010  with  mature  critical  technologies 
and,  according  to  program  office  metrics,  a  stable 
design,  but  before  bringing  manufacturing 
processes  under  control.  A  recent  assessment  of 
the  program's  manufacturing  readiness  levels 
indicates  that  these  processes  are  now 
demonstrated  and  under  control  as  the  program 
approaches  full  rate  production.  However,  during 
its  May  2012  initial  operational  test,  the  program 
did  not  demonstrate  required  performance  and 
reliability.  A  follow-on  operational  test  in  May  2013 
showed  improved  reliability  and  performance  but 
revealed  that  deficiencies  remain.  Consequently, 
the  milestone  decision  authority  has  delayed  the 
full  rate  production  decision  until  the  Army 
completes  an  additional  follow-on  operational  test 
and  confirms  that  deficiencies  have  been 
corrected. 
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Attainment  of  Product  Knowledge 


As  of  January  2014 


Resources  and  requirements  match 


Demonstrate  all  critical  technologies  in  a  relevant 
environment 


Demonstrate  all  critical  technologies  in  a  realistic 
environment 


Complete  preliminary  design  review 


Product  design  is  stable 


•  Release  at  least  90  percent  of  design  drawings 

•  Test  a  system-level  integrated  prototype 


Manufacturing  processes  are  mature 


•  Demonstrate  critical  processes  are  in  control 

•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


9  Knowledge  attained 
o  Knowledge  not  attained 


Information  not  available 

Not  applicable 
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Common  Name:  WIN-T  Increment  2 


WIN-T  Increment  2  Program 

Technology  Maturity 

All  15  WIN-T  Increment  2  critical  technologies  were 
mature  by  its  February  2010  production  decision.  In 
August  2012,  an  independent  manufacturing 
readiness  assessment  prepared  by  the  Army 
concluded  that  the  prime  contractor  had  achieved 
an  acceptable  level  of  technology  maturity  to 
continue  to  full  rate  production. 

Design  Maturity 

According  to  the  WIN-T  program,  it  has  integrated 
and  tested  its  key  technologies  and  subsystems  to 
demonstrate  that  the  system's  design  is  capable  of 
working  as  intended.  The  program  office  does  not 
track  the  metric  we  use  to  measure  design 
maturity — the  number  of  releasable  drawings — as 
WIN-T  is  primarily  an  information  technology 
integration  effort.  Instead,  design  performance  is 
measured  through  a  series  of  component, 
subsystem,  configuration  item,  and  network-level 
tests  designed  to  demonstrate  performance  at 
increasing  levels  of  system  integration.  Design 
stability  is  measured  through  problem-tracking 
report  trends. 

Production  Maturity 

The  WIN-T  Increment  2  program  began  production 
in  February  2010  and  began  testing  a  production- 
representative  prototype  13  months  later  in  March 
2011.  The  program  indicates  that  its  manufacturing 
processes — as  determined  by  an  Army 
manufacturing  readiness  assessment — are  now  in 
control,  but  had  not  yet  been  demonstrated  or 
brought  in  control  at  production  start.  During  its  May 
2012  initial  operational  test,  the  program  did  not 
demonstrate  required  performance  and  reliability.  A 
follow-on  operational  test  in  May  2013  showed 
improved  reliability  and  performance  but  revealed 
that  deficiencies  remain.  In  particular,  three  of  the 
program's  nine  configuration  items  were  assessed 
as  not  operationally  effective  and  two  were  not 
operationally  suitable  due  to  complexity  of 
operations  and  reliability  problems.  In  addition,  the 
program  demonstrated  cyber  security  vulnerabilities 
that  need  improvement.  Consequently,  the 
milestone  decision  authority  has  delayed  a  full  rate 
production  decision  until  the  Army  completes  an 
additional  follow-on  operational  test  in  November 
2014  as  part  of  the  Army's  Network  Integration 


Evaluation  15.1  that  confirms  that  deficiencies  have 
been  corrected.  A  full-rate  production  decision  is 
planned  for  May  2015. 

Other  Program  Issues 

While  the  milestone  decision  authority  has  not  yet 
approved  a  full-rate  production  decision,  the  Army 
declared  that  the  program  attained  its  initial 
capability  in  August  2013.  This  decision  was  based 
on  the  Army's  assertion  that  initial  operational 
capability  was  reached  when  the  first  brigade 
combat  team  and  its  division  headquarters  was 
equipped  and  fully  trained  with  the  new  equipment 
and  the  follow  on  operational  test  had  been 
successfully  completed.  However,  as  noted  above, 
the  May  2013  follow-on  operational  test  revealed 
deficiencies  with  the  program  that  must  be 
corrected  and  these  corrections  must  be  confirmed 
during  an  additional  follow-on  operational  test. 

Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Army  noted  that  in  September  2013,  it  received 
approval  from  the  Defense  Acquisition  Board  for  an 
additional  low-rate  initial  production  lot  and  an 
acceptance  to  field  existing  lots  of  WIN-T  Increment 
2.  The  Army  has  fielded  five  Brigade  Combat  Teams 
and  two  divisions,  and  remains  on  track  to  field 
Capability  Set  14  to  units,  including  Stryker  Brigade 
Combat  Teams.  Using  soldier  feedback  from  theater 
and  test  results,  the  Army  intends  to  continue  to 
improve  WIN-T  Increment  2  capabilities  with  the 
primary  focus  being  on  simplifying  the  operations 
and  maintenance  of  the  equipment.  The  program 
office  also  provided  technical  comments,  which 
were  incorporated  where  deemed  appropriate. 
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Common  Name:  WIN-T  Increment  3 


Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 


WIN-T  is  the  Army's  high-speed  and  high-capacity 
backbone  communications  network.  It  connects 
units  with  higher  levels  of  command  and  provides 
the  Army's  tactical  portion  of  the  Global  Information 
Grid.  WIN-T  was  restructured  following  a  March 
2007  Nunn-McCurdy  unit-cost  breach  of  the  critical 
threshold,  and  will  be  fielded  in  four  increments.  We 
assessed  the  third  increment,  which  is  expected  to 
provide  the  Army  a  full  networking  on-the-move 
capability. 


Source:  U.S.  Army. 


ancept  System  development 


Program/ 
development  start 
(7/03) 


Program  Essentials 

Prime  contractor:  General  Dynamics  C4 
Systems,  Inc. 

Program  office:  Aberdeen  Proving 
Ground,  MD 

Funding  needed  to  complete: 

R&D:  $728.0  million 
Procurement:  $12,850.7  million 
Total  funding:  $13,578.7  million 
Procurement  quantity:  3,449 


► 


▲  ▲ 

Design  GAO  Low-rate 

review  review  decision 

(12/13)  (1/14)  (TBD) 

▲ 

Start 

operational  test 
(TBD) 

▲ 

Full-rate 

decision 

(TBD) 

▲ 

Initial 

capability 

(TBD) 

Program  Performance  (fiscal  year  2014  dollars  in  millions) 

As  of  Latest  Percent 

05/2009  08/2013  change 

Research  and  development  cost 

$2,826.4 

$2,563.0 

-9.3 

Procurement  cost 

$14,387.9 

$12,850.7 

-10.7 

Total  program  cost 

$17,214.3 

$15,413.7 

-10.5 

Program  unit  cost 

$4,944 

$4,388 

-11.3 

Total  quantities 

3,482 

3,513 

0.9 

Acquisition  cycle  time  (months) 

165 

199 

20.6 

The  Army  is  restructuring  the  program.  Changes  to  the  program's  costs,  quantities,  and  schedule 
from  this  restructuring  are  not  reflected  in  the  latest  column  above. 

According  to  an  Army  representative,  1 2  of  Wl  N-T 
Increment  3's  18  current  critical  technologies  are 
mature  and  6  are  nearing  maturity  following  the 
critical  design  review  of  the  program  in  December 
2013.  However,  the  program  is  being  restructured 
based  on  the  recommendations  of  a  configuration 
steering  board  held  in  November  2013.  This 
restructuring  could  affect  the  program's  need  for 
certain  critical  technologies.  The  program  does 
not  use  the  number  of  design  drawings  released 
or  alternative  methods  to  assess  design  stability. 
However,  the  program  plans  to  begin  capturing 
software  design  stability  metrics  once  a  baseline 
system  design  is  established. 


•  Demonstrate  critical  processes  are  in  control 


•  Demonstrate  critical  processes  on  a  pilot  production  line 

•  Test  a  production-representative  prototype 


^  Knowledge  attained  ■■■■  Information  not  available 

o  Knowledge  not  attained  Not  applicable 
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Common  Name:  WIN-T  Increment  3 


WIN-T  Increment  3  Program 

Technology  Maturity 

According  to  an  Army  representative,  12  of  the 
program's  18  current  critical  technologies  are 
mature,  and  6  are  nearing  maturity  following  the 
critical  design  review  of  the  program  in  December 
2013.  However,  in  January  2014  the  Army 
acquisition  executive  directed  the  program 
executive  officer  with  oversight  of  the  WIN-T 
Increment  3  program  to  restructure  the  program 
based  on  the  recommendations  of  the  Army's 
configuration  steering  board. 

The  program's  restructuring  could  affect  its  need  for 
certain  critical  technologies.  For  example,  the 
Army's  configuration  steering  board  has 
recommended  descoping  the  air  tier  from  the 
program.  According  to  an  Army  representative,  the 
program  will  need  to  complete  a  follow-on  critical 
design  review  once  it  has  been  restructured,  at 
which  point  the  critical  technologies  will  need  to  be 
reassessed. 

Design  Maturity 

The  program  does  not  use  the  number  of  design 
drawings  released  or  any  alternative  methods  to 
assess  design  stability.  According  to  the  program 
office,  the  number  of  design  drawings  is  not 
meaningful  as  WIN-T  is  not  a  manufacturing  effort, 
but  a  software  development  and  hardware 
integration  effort.  The  program  asserts  that  it  has 
appropriate  software  metrics  and  tracks  design 
stability  through  its  configuration  management 
software  production  plan. 

Other  Program  Issues 

According  to  an  Army  representative,  the  Army  is  in 
the  process  of  conducting  a  "network  capability" 
review  to  determine  the  "right  mix"of  tactical 
networking  capabilities  to  provide  to  brigade  combat 
teams.  In  addition,  the  program  is  awaiting  the  final 
assessments  from  the  Army's  recently  completed 
configuration  steering  board.  These  two  events  will 
factor  into  the  scope  of  the  restructured  program. 
According  to  an  Army  representative,  restructuring 
considerations  may  include  maximizing  the  use  of 
the  capabilities  currently  being  fielded  under  the 
WIN-T  Increment  2  program  while  maintaining  focus 
on  WIN-T  Increment  3  network  operations  efforts 
and  balancing  affordability,  as  DOD  and  the  Army 
face  long-term  budgetary  challenges. 


Program  Office  Comments 

In  commenting  on  a  draft  of  this  assessment,  the 
Army  noted  that  as  of  early  2014,  the  WIN-T 
Increment  3  program  is  anticipating  a  final  program 
assessment  from  the  Department  following  the 
program's  critical  design  review  and  is  also  awaiting 
Department's  approval  of  a  program  transition 
strategy  based  on  the  WIN-T  Increment  3 
configuration  steering  board  meeting  held  in 
November  2013.  These  events,  coupled  with 
ongoing  Army  tactical  network  capability  review 
decisions,  will  provide  guidance  on  the  scope  of  the 
program's  effort  going  forward. 
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Common  Name:  AMF  JTRS 


Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System  (AMF  JTRS) 

The  Army's  AMF  JTRS  program  plans  to  acquire 
two  non-developmental  software-defined  radios,  the 
Small  Airborne  Link  16  Terminal  (SALT)  and  the 
Small  Airborne  Networking  Radio  (SANR),  and 
associated  equipment  for  integration  into  Army 
rotary  wing  and  unmanned  aerial  systems. 

Previously,  the  program  had  been  developing 
software-defined  radios  and  associated  equipment 
for  integration  into  nearly  160  different  types  of 
aircraft,  ships,  and  fixed  stations  to  increase 
communications  and  networking  capabilities. 


Current  Status 

The  Linder  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics,  in  July  201 2,  restructured  the 
AMF  JTRS  program  due  to  concerns  about  the  potential  for  unbounded  cost  increases,  and  continued 
schedule  and  contractor  performance  risk.  Market  research  also  determined  that  non-developmental  items 
could  be  modified  to  meet  users'  revised  needs  and  priorities.  The  Under  Secretary  also  transferred 
responsibility  for  program  management  to  the  Army.  While  the  use  of  a  non-developmental  item  approach 
will  likely  reduce  some  technical  risk,  the  Army  will  need  to  manage  the  integration  of  government- 
developed  waveforms  into  the  non-developmental  item  radios  and  the  radios  into  various  platforms. 

AMF  JTRS  now  plans  to  acquire  two  non-developmental  software-defined  radios.  The  SALT  is  designed  to 
be  a  2-channel  radio  capable  of  running  the  Link  16  waveform  and  the  Soldier  Radio  Waveform  (SRW).  The 
program  plans  to  introduce  production  hardware  into  the  Apache  AH-64E  assembly  line  in  fiscal  year  2016. 
Until  then,  according  to  program  officials,  the  Apache  program  office  is  pursuing  an  interim  solution,  the 
Small  Tactical  Terminal  (STT),  which  will  run  only  Link  16.  The  Army  acquisition  strategy  for  SALT  calls  for 
full  and  open  competition.  The  Army  plans  to  release  the  SALT  request  for  proposals  in  the  second  quarter 
of  fiscal  year  2014  and  a  full  rate  production  decision  is  scheduled  for  fiscal  year  2016. 

The  SANR  is  designed  as  a  2-channel  radio  capable  of  running  the  SRW,  Single  Channel  Ground  and 
Airborne  Radio  System  waveform,  and  the  Wideband  Networking  Waveform.  The  SANR  acquisition  strategy 
plans  a  full  and  open  competition  to  select  two  contractors  for  SANR  test  assets.  A  low  rate  initial  production 
decision  is  scheduled  for  fiscal  year  2016  and  full  rate  production  is  scheduled  for  fiscal  year  2018. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $3,582.7  million 
Research  and  development:  $1,849.3  million 
Procurement:  $1,733.4  million 
Quantity:  15,652  channels 

Next  Major  Program  Event:  SALT  full-rate  production  and  SANR  low-rate  initial  production  decisions  in  first 
quarter  2016. 

Program  Office  Comments:  The  program  office  concurred  with  the  assessment. 


Source:  U.S.  Army. 
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Common  Name:  ACV 


Amphibious  Combat  Vehicle  (ACV) 

The  Marine  Corps'  ACV  is  intended  to  transport 
troops  from  ship  to  shore  and  secure  a  beachhead. 

The  Marine  Corps  has  a  requirement  to  replace  the 
existing  amphibious  assault  vehicle  (AAV)  but  the 
Secretary  of  Defense  terminated  the  initial  effort,  the 
Expeditionary  Fighting  Vehicle  (EFV)  program,  in 
January  2011  due  to  technology  problems, 
development  delays,  and  cost  increases. 


Current  Status 

A  December  2011  materiel  development  decision  memorandum  directed  that  a  "highly  tailored"  approach  be 
used  to  acquire  the  ACV  and  that  this  effort  focus  on  a  cost-effective  solution,  emphasizing  engineering  and 
design  analysis  through  establishment  of  affordability  targets.  According  to  officials,  the  Marine  Corps 
completed  an  ACV  analysis  of  alternatives  (AOA)  in  July  201 2  and  the  Navy  forwarded  it  to  the  Office  of  the 
Secretary  of  Defense.  According  to  ACV  program  officials,  although  high-water  speed  had  been  a  primary 
requirement  for  the  EFV,  it  was  not  part  of  the  initial  ACV  AOA  study.  Instead,  the  study  was  based  on  a 
speed  requirement  that  was  about  half  of  the  EFV's,  which  would  make  the  ACV  a  "displacement"  vehicle 
more  similar  in  water  speed  to  the  AAV.  Marine  Corps  leadership  subsequently  directed  that  the  affordability 
of  a  high-water  speed  amphibious  vehicle  be  studied  before  the  ACV  began  acquisition.  This  feasibility 
analysis  is  ongoing. 

Officials  reported  that  the  Marine  Corps  is  studying  trade-offs  in  ACV  capability  to  find  the  ACV  design  that 
will  provide  improvements  in  amphibious  capability-speed,  range,  lethality,  and  survivability-over  the  current 
amphibious  vehicle  at  an  affordable  cost.  They  added  that,  following  the  feasibility  analysis,  the  Marine 
Corps  Commandant  will  make  a  decision  on  whether  the  ACV  acquisition  program  will  formally  begin  at 
technology  development  or  system  development.  Expectations  are  that  the  ACV  will  not  reach  initial 
operating  capability  before  2023.  To  address  the  delay  in  fielding  an  amphibious  vehicle  to  replace  the 
legacy  AAVs,  392  current  AAVs  will  undergo  survivability  upgrades  that  will  address  deficiencies  in 
survivability. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012-2017):  $1,047.1  million 
Procurement:  TBD 

Acquisition  related  operation  and  maintenance  (fiscal  years  2012-2017):  $18.9  million 
Quantity:  TBD 

Next  Major  Program  Event:  System  development  start,  TBD 

Program  Office  Comments:  The  Marine  Corps  was  provided  a  draft  of  this  assessment  and  did  not  offer 
any  comments. 


Source:  U.S.  Marine  CorDS. 
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Common  Name:  AMPV 


Armored  Multi-Purpose  Vehicle  (AMPV) 

The  Army's  Armored  Multi-Purpose  Vehicle  (AMPV) 
fleet  is  the  proposed  replacement  to  the  Ml  13  family 
of  vehicles  in  the  armored  brigade  combat  team. 

The  AMPV  will  replace  the  Ml  13  in  five  mission 
roles:  general  purpose,  medical  evacuation,  medical 
treatment,  mortar  carrier,  and  mission  command. 

The  Army  has  determined  that  development  of  the 
AMPV  is  necessary  due  to  mobility  deficiencies 
identified  in  the  Ml  13,  as  well  as  space,  weight, 
power  and  cooling  limitations  that  prevent  the 
incorporation  of  future  technologies. 


Current  Status 

Based  on  direction  at  the  August  and  November  2013  defense  acquisition  board  reviews,  AMPV's 
acquisition  strategy  was  updated  and  approved  before  a  request  for  proposals  was  released  in  November 
2013.  The  program's  acquisition  strategy  is  based  on  modifying  an  existing  platform  and  bypasses  the 
technology  development  phase  to  begin  in  system  development.  The  program  is  still  working  to  define  key 
technologies,  and  as  a  result,  will  require  potential  contractors  to  include  data  related  to  the  technology 
maturity  of  their  design  in  each  proposal.  AMPV's  acquisition  strategy  also  calls  for  selection  of  a  single 
contractor  for  system  development,  and  provides  for  three  low-rate  initial  production  options  within  the 
system  development  contract.  The  program  currently  plans  to  award  a  cost  plus  incentive  fee  contract  for 
system  development  with  a  fixed  price  incentive  option  for  low-rate  initial  production.  The  AMPV  contract  will 
include  engineering  and  manufacturing  development  and  low-rate  initial  production  incentives  for  cost  and 
reliability. 

Procurement  of  AMPV,  with  an  estimated  total  acquisition  cost  of  up  to  $10.2  billion,  will  occur  at  about  the 
same  time  as  the  Army's  Joint  Light  Tactical  Vehicle  program.  The  procurement  of  these  programs  is 
expected  to  continue  for  a  decade  or  more. 

Estimated  Total  Program  Cost  and  Quantity  (fiscal  year  2014  dollars) 

Total  Program:  $10,223  million 
Research  and  development:  $779.9  million 
Procurement:  $9,443.0  million 
Quantity:  2,907 

Next  Major  Program  Event:  Start  of  system  development,  first  quarter  fiscal  year  2015 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  B-2  DMS 


B-2  Defensive  Management  System  Modernization  (B-2  DMS) 

The  Air  Force's  B-2  DMS  modernization  program  is 
expected  to  upgrade  the  aircraft's  analog  defensive 
management  system  to  a  digital  capability  to  detect, 
identify  and  avoid  threats.  The  modernization  is 
intended  to  update  pilot  displays  and  enhance  the 
in-flight  capability  to  avoid  unanticipated  air  defense 
threats.  Improvement  in  frequency  coverage, 
sensitivity  of  the  electronic  warfare  suite,  and 
reliability  and  maintainability  are  also  anticipated. 

The  latter  is  expected  to  improve  the  B-2's 
readiness  rate. 


Current  Status 

In  October  2012,  the  program  entered  a  second  technology  development  phase,  which  continued 
throughout  2013.  The  program  had  scheduled  a  preliminary  design  review  prior  to  beginning  system 
development  in  April  2014,  but  officials  told  us  they  are  currently  restructuring  the  program  due  to  expected 
future  budget  reductions.  Program  documentation  indicates  moderate  technical  risk  due  to  the  program's 
schedule  and  nuclear  hardening  activities.  Also,  the  antenna  design,  the  software  schedule,  and  the 
availability  of  test  aircraft  are  risk  areas.  Early  in  the  program,  an  independent  assessment  identified  six 
critical  technologies;  however,  currently  the  program  does  not  believe  any  critical  technologies  exist. 
Another  independent  technology  readiness  assessment,  scheduled  prior  to  system  development,  will 
identify  the  final  list  of  critical  technologies. 

We  have  previously  reported  that  the  program  was  implementing  a  rapid  acquisition  initiative  to  reduce  its 
acquisition  time  by  up  to  3  years  and  lower  costs  by  as  much  as  $500  million.  To  achieve  these  results, 
several  activities  were  planned  including:  early  software  prototyping,  reducing  the  time  required  for  flight 
testing,  and  improving  antenna  installation  times.  By  conducting  these  activities,  it  was  anticipated  the 
program  might  take  7  years  to  complete  and  cost  $1.58  billion.  While  some  parts  of  the  initiative  have 
yielded  positive  results,  including  a  subcontractor  competition  for  antenna  development  that  officials  told  us 
saved  time  and  money,  the  program's  start  of  development,  flight  testing,  and  full  operational  capability  have 
all  been  delayed.  The  program  now  expects  to  complete  acquisition  in  almost  9  years  at  a  cost  of  over  $2 
billion,  which  means  most  of  the  expected  reductions  in  cost  and  schedule  from  the  initiative  will  not  be 
realized. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $2,036  million 
Research  and  development:  $1,385  million 
Procurement:  $651  million 
Quantity:  20 

Next  Major  Program  Event:  System  development  start,  April  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  DOD  officials  told  us  the 
assessment  is  accurate  based  on  the  fiscal  year  2014  budget.  They  added  that  the  B-2  DMS  modernization 
program  remains  the  top  B-2  modernization  priority.  The  program  office  also  provided  technical  comments 
which  were  incorporated  as  appropriate. 


Source:  U.S.  Air  Force. 
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Common  Name:  CRH 


Combat  Rescue  Helicopter  (CRH) 

The  Air  Force's  Combat  Rescue  Helicopter  (CRH) 
program,  formerly  called  HH-60  Recapitalization,  is 
an  effort  to  replace  aging  HH-60G  Pave  Hawk 
helicopters.  The  CRH's  primary  mission  is  to 
recover  personnel  from  hostile  or  denied  territory;  it 
will  also  conduct  humanitarian,  civil  search  and 
rescue,  disaster  relief,  and  non-combatant 
evacuation  missions.  The  program  is  the  second 
effort  to  replace  the  HH-60G.  The  first,  the  Combat 
Search  and  Rescue  Replacement  Vehicle  (CSAR- 
X),  was  canceled  because  of  cost  concerns  in  2009. 


Current  Status 

The  Air  Force  expects  that  the  CRH  will  be  an  existing  helicopter  with  modifications  to  integrate  mature 
technology  subsystems  and  associated  software.  The  CRH  program  received  its  materiel  development 
decision  in  March  2012  and  DOD  authorized  the  program  to  bypass  technology  development  and  enter  the 
acquisition  process  at  system  development.  The  program  successfully  completed  a  defense  acquisition 
board  review  in  September  2012  which  approved  its  acquisition  strategy  and  release  of  a  request  for 
proposals  for  a  full  and  open  competition  in  October  2012.  Depending  upon  the  availability  of  future  funding, 
the  CRH  program  office  plans  to  enter  system  development  in  March  2014,  delayed  from  the  originally 
scheduled  August  2013.  The  program  is  preparing  to  then  award  a  fixed  price  incentive  firm  target 
development  contract  to  Sikorsky,  which  was  the  only  contractor  who  responded  to  the  request  for 
proposals.  However,  the  program  does  not  intend  to  conduct  any  systems  engineering  technical  reviews 
until  after  the  development  contract  is  awarded.  According  to  officials  from  the  Office  of  the  Secretary  of 
Defense,  the  program  was  granted  a  waiver  in  late  2012  to  conduct  its  preliminary  design  review  after  the 
start  of  system  development,  as  well  as  a  waiver  for  the  requirement  to  conduct  competitive  prototyping.  In 
our  previous  work,  we  have  found  that  acquisition  programs  which  successfully  complete  robust  systems 
engineering  early  in  the  acquisition  process  and  conduct  a  preliminary  design  review  prior  to  starting  system 
development  typically  have  better  outcomes.  Initial  operational  capability  for  the  helicopter  is  expected  in  the 
fourth  quarter  of  fiscal  year  2019. 

Estimated  Total  Program  Cost  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2012-2018):  $1,310.9  million 

Procurement  (fiscal  years  2016-2018):  $  2,187.7  million 

Military  construction:  TBD 

Acquisition  operations  and  maintenance:  TBD 

Total  quantity:  112 

Next  Major  Program  Event:  System  development  start,  March  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 


Source:  2013  Sikorsky  Aircraft  Corp.  Used  with  permission  for  support  of  CRH. 
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Common  Name:  CIRCM 


Common  Infrared  Countermeasure  (CIRCM) 

The  Army's  CIRCM,  the  next  generation  of  the 
Advanced  Threat  Infrared  Countermeasures 
(ATIRCM),  will  be  used  with  a  missile  warning 
system  and  a  countermeasure  dispenser  capable  of 
employing  expendables,  such  as  flares  and  chaff,  to 
defend  aircraft  from  infrared-guided  missiles.  The 
CIRCM  program  will  develop  a  laser-based 
countermeasure  system  for  rotary-wing,  tilt-rotor, 
and  small  fixed-wing  aircraft  across  DOD.  CIRCM  is 
one  of  three  subprograms  that  make  up  the 
ATIRCM/CMWS  major  defense  acquisition 
program. 


Current  Status 

The  CIRCM  program  began  in  2009  when  the  Under  Secretary  of  Defense  for  Acquisition,  Technology  and 
Logistics  supported  the  Army's  decision  to  restructure  the  ATIRCM/CMWS  program.  In  June  2009,  the  Army 
received  approval  to  award  five  contracts  to  provide  prototype  systems  for  testing.  After  testing  these 
prototypes,  the  Army  concluded  that  the  systems  were  not  mature  enough  for  entry  into  system 
development.  The  Army  subsequently  decided  that  the  program  should  proceed  with  a  technology 
development  phase  that  will  include  additional  prototyping  efforts  to  further  mature  CIRCM  technologies  and 
awarded  two  contracts  in  January  2012. 

The  Army  held  preliminary  design  reviews  in  July  and  August,  2013  with  both  contractors.  The  review  report 
stated  that  the  system's  preliminary  design  satisfies  the  operational  and  suitability  requirements.  Also,  the 
Army  recently  conducted  a  preliminary  technology  readiness  assessment  based  on  contractor  testing  in 
which  nine  critical  technology  elements  were  identified  and  are  approaching  full  maturity  leading  up  to 
system  development.  A  final  technology  readiness  assessment  will  be  conducted  prior  to  development  to 
verify  technology  and  design  maturity  and  will  be  based  on  government-witnessed  testing. 

In  October  2013,  the  two  CIRCM  contractors  completed  prototypes  deliveries.  According  to  the  Army,  both 
designs  are  mature  and  are  low  risk  of  not  meeting  the  specified  requirements  even  with  the  current 
configurations.  The  Army  plans  to  make  a  system  development  decision  in  December  2014  and  plans  to 
award  one  contractor  a  cost  plus  fixed  fee/firm  fixed  price  contract  with  technical  performance  incentives 
focused  on  achievement  of  weight  and  reliability  goals. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  Program:  $3,599.8  million 
Research  and  development:  $788.8  million 
Procurement:  $2,811.0  million 
Quantity:  1,076 

Next  Major  Program  Event:  System  development  start,  December  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 


Sources:  ©  2011  BAE  Systems  (left);  Northrop  Grumman  (right) 
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Common  Name:  DDG  51 


DDG  51  Destroyer  (DDG  51) 

The  DDG  51  destroyer  is  a  multimission  ship 
designed  to  operate  against  air,  surface,  and 
subsurface  threats.  After  a  nearly  4-year  break,  the 
Navy  restarted  Flight  IIA  production  and  plans  to 
buy  at  least  nine  ships  between  fiscal  years  2013 
and  2017.  The  Navy  plans  to  procure  three  Flight  III 
ships,  a  modification  of  the  Flight  IIA  design, 
beginning  in  fiscal  year  2016.  Flight  III  is  expected 
to  have  an  increased  focus  on  missile  defense  and 
will  include  the  Air  and  Missile  Defense  Radar 
(AMDR),  initially  an  S-band  radar  developed  under 
a  separate  program. 

Current  Status 

Four  DDG  51  Flight  IIA  ships  are  under  construction  and  an  additional  nine  ships  are  under  contract.  The 
design  of  the  Flight  IIA  ships  will  be  modified  to  include  an  upgraded  Aegis  combat  system  currently  being 
developed.  The  Navy  will  also  replace  the  existing  SPS-67  radar  with  SPQ-9B,  currently  installed  on  several 
Navy  ships,  beginning  in  fiscal  year  2014.  The  Navy  believes  that  it  has  largely  resolved  issues  with 
production  of  a  new  reduction  gear. 

DDG  51  Flight  III  ships  are  expected  to  feature  new  electric  plants,  new  air-conditioning  plants,  and  the 
AMDR.  According  to  the  Navy,  the  new  electric  plants  are  based  on  a  design  used  on  DDG  1000  and 
modification  will  be  required  for  integration  with  DDG  51.  The  DDG  1000  electrical  system  has  faced  delays 
in  completing  testing.  Detail  design  work  for  Flight  III  will  begin  at  the  end  of  fiscal  year  2014,  according  to 
the  Navy.  Adding  AMDR  to  DDG  51  will  result  in  a  significant  redesign  of  the  ship  and  the  Navy  expects  that 
Flight  III  will  result  in  changes  to  more  than  25  percent  of  Flight  IIA  drawings,  although  the  Navy  believes 
many  of  these  will  be  minor  alterations.  The  Navy  will  need  AMDR's  design  assumptions,  such  as  its  size, 
shape,  weight,  and  power  and  cooling  requirements  in  order  to  accurately  redesign  the  ship.  However,  the 
Navy  only  recently  awarded  a  contract  for  AMDR  system  development  and  the  AMDR  program  is  at  least  6 
months  behind  schedule.  Based  on  its  current  schedule,  the  Navy  plans  to  begin  detail  design  work  for 
Flight  III  at  the  end  of  fiscal  year  2014 — before  AMDR  has  demonstrated  full  maturity-adding  risk  and 
uncertainty  to  the  DDG  51  program. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program  (fiscal  years  2010  to  2018):  $23,089.5  million 
Research  and  development  (fiscal  years  2010  to  2018):  $900  million 
Procurement  (fiscal  years  2010  to  2018):  $22,189.6  million 
Quantity:  15 

Next  Major  Program  Event:  Defense  acquisition  board  review  of  Flight  III  modifications,  third  quarter  fiscal 
year  2014. 

Program  Office  Comments:  According  to  program  officials,  the  Flight  III  schedule  integrates  all  equipment 
development  efforts  into  the  overall  ship  detail  design  effort.  This  is  the  same  process  successfully 
implemented  on  previous  flight  upgrades.  Due  to  the  AMDR  technology  development  phase,  which 
produced  working  1000-element  arrays,  the  current  design  maturity  supports  commencement  of  detail 
design.  Flight  III  is  on  track  to  be  implemented  on  the  second  fiscal  year  2016  ship. 


Source:  Bath  Iron  Works. 
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Common  Name:  F-15  EPAWSS 


F-15  Eagle  Passive/Active  Warning  and  Survivability  System  (F-15  EPAWSS) 

The  Air  Force's  F-15  EPAWSS  program  is  intended 
to  upgrade  the  electronic  warfare  system  on  fielded 
F-15  aircraft.  The  program  seeks  to  improve  the 
aircraft's  internal  self-protection  electronic  warfare 
systems  to  enable  operations  in  current  and  future 
threat  environments.  The  program  is  also  expected 
to  improve  the  F-15's  survivability  by  enhancing  its 
ability  to  detect,  identify,  locate,  deny,  degrade, 
disrupt,  and  defeat  air  and  ground  threat  systems. 


Current  Status 

In  November  2012,  the  Office  of  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics 
approved  the  Air  Force's  plans  to  conduct  an  analysis  of  alternatives  and  to  award  a  pre-engineering  and 
manufacturing  development  characterization  contract  for  the  F-15  EPAWSS  program,  which  occurred  in 
fiscal  year  2013.  The  Air  Force  plans  to  leverage  non-developmental  electronic  warfare  technologies  and 
components,  currently  used  in  other  Air  Force  and  Navy  aircraft,  to  create  a  state-of-the-art  wide  bandwidth 
digital  electronic  warfare  system  capable  of  protecting  the  F-15  aircraft  against  advanced  enemy  threats. 
The  Air  Force  is  currently  updating  the  program's  requirements  documentation  and  intends  to  conduct  a 
preliminary  design  review  before  starting  system  development  in  fiscal  year  2016.  The  program  office  plans 
to  award  a  production  contract  in  fiscal  year  2018. 

An  early  Air  Force  assessment  of  F-15  EPAWSS  identified  digital  receiver  technology — both  the  hardware 
and  the  associated  software — as  a  critical  technology  that  the  program  does  not  intend  to  fully  mature  prior 
to  the  start  of  system  development.  According  to  program  officials,  a  key  risk  is  integration  of  the  electronic 
warfare  system  with  all  other  on-board  and  off-board  systems,  so  that  it  operates  properly  in  its  intended 
operational  environment.  The  program  expects  to  add  future  capabilities  as  new  electronic  warfare  threats 
emerge. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development:  $614.9  million  (fiscal  years  2014  to  2018) 

Procurement:  TBD 
Quantity:  392 

Next  Major  Program  Event:  Defense  acquisition  board  review  of  acquisition  strategy,  February  2014 

Program  Office  Comments:  According  to  the  Air  Force,  the  program  office  continues  to  refine  the  planning 
for  the  F-15  EPAWSS  program  based  on  in-work  requirements  documentation  and  final  acquisition  strategy 
approval  by  the  Milestone  Decision  Authority. 


Source:  U.S.  Air  Force. 
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Common  Name:  GCV 


Ground  Combat  Vehicle  (GCV) 

The  Ground  Combat  Vehicle  (GCV)  is  an 
incremental  program  to  replace  segments  of  the 
Army's  combat  vehicle  inventory.  The  first  variant  is 
intended  to  be  the  service's  next  infantry  fighting 
vehicle,  replacing  a  portion  of  the  current  M2 
Bradley  fleet.  The  Army  expects  GCV  to  provide  a 
full-spectrum  capability  to  perform  offensive, 
defensive,  stability,  and  support  operations;  carry  a 
nine-soldier  squad;  emphasize  force  protection;  and 
be  operational  within  7  years  of  beginning 
technology  development. 


Current  Status 

In  August  2011 ,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  GCV's 
entry  into  technology  development.  The  Army  awarded  contracts  to  two  contractor  teams,  but  resolving  a  bid 
protest  prevented  the  start  of  technology  development  until  December  2011.  Since  then,  the  Army  has  been 
involved  in  three  activities:  updating  the  analysis  of  alternatives,  assessing  upgrades  to  existing  vehicles, 
and  funding  two  contractors'  efforts  to  build  and  demonstrate  key  subsystem  prototypes. 

On  January  16,  2013,  the  Under  Secretary  directed  a  number  of  changes  to  the  program,  including 
extending  the  technology  development  phase  six  months,  delaying  both  system  development  and 
production,  and  directing  that  a  single  prime  contractor  be  selected  for  system  development.  These  actions 
provide  significant  reductions  to  the  funding  necessary  to  execute  the  program. 

An  independent  review  team  identified  three  candidate  critical  technologies — all  nearing  full  maturity — that 
could  be  incorporated  into  the  GCV  design.  The  program  will  continue  monitoring  these  technologies  and 
consider  their  maturity  and  risk  as  part  of  the  full  and  open  competition  planned  for  system  development. 
Since  our  last  report,  both  contractors  continued  technology  development  by  maturing  preliminary  designs 
and  subsystems,  completing  multiple  tests  of  their  respective  mine  blast  protection  designs,  and  testing 
engines  and  drive  trains.  After  holding  preliminary  design  reviews  to  determine  whether  the  proposed 
designs  could  satisfy  requirements,  a  defense  acquisition  board  will  determine  the  program's  readiness  to 
proceed  into  system  development,  which  is  expected  to  begin  in  June  2014. The  Army  plans  to  begin 
procuring  GCV  while  simultaneously  procuring  other  new  and  costly  combat  vehicles  including  the  Joint 
Light  Tactical  Vehicle. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $29,219.6  million 

Research  and  development:  $4,968.5  million 

Procurement:  $24,229.1  million 

Quantity:  12  (development),  1,874  (procurement) 

Next  Major  Program  Event:  System  development  start,  June  2014 

Program  Office  Comments:  According  to  officials  in  the  office  of  the  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics,  the  GCV  program  was  cancelled  on  the  order  of  the  Secretary  of 
Defense. 


VEHICLE 


Source:  U.S.  Army. 
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Common  Name:  IFPCInc2 


Indirect  Fire  Protection  Capability-Increment  2  (IFPC  Inc  2) 

The  Army's  IFPC  Increment  2  consists  of  three 
blocks,  each  a  separate  major  defense  acquisition 
program,  to  detect,  assess,  and  defend  against 
threats  from  rockets,  artillery,  mortars,  cruise 
missiles,  and  unmanned  aircraft.  The  first  IFPC 
increment,  fielded  in  2004,  provides  a  short-range 
capability  to  counter  rockets,  artillery,  and  mortar 
threats.  IFPC  Increment  2,  Block  1  establishes  the 
capability  to  counter  cruise  missiles  and  unmanned 
aircraft.  The  remaining  blocks  add  to  and  extend  the 
range  and  capability. 


Current  Status 

In  2004,  after  the  first  increment  of  IFPC  was  fielded,  users  developed  a  new  requirement  to  counter 
unmanned  aircraft  and  cruise  missiles.  The  Surface  Launched  Advanced  Medium-Range  Air-to-Air  Missile 
was  originally  set  to  fulfill  this  requirement,  but  was  cancelled.  To  replace  it,  in  2011 ,  the  Army  proposed  a 
new  interceptor,  launcher,  and  radar.  The  $1 .6  billion  estimate  for  developing  this  new  system,  however,  was 
deemed  unaffordable.  The  Army  then  proposed  a  three  block  alternative,  each  a  separate  major  defense 
acquisition  program,  which  leverages  an  existing  interceptor  and  sensor  and  competitively  awards  the 
development  of  a  new  launcher.  Block  1  is  scheduled  for  fielding  in  fiscal  year  201 9.  The  review  to  begin 
technology  development  for  Block  1  was  held  October  22,  2013,  but  final  approval  is  pending  until  January 
2014  due  to  budget  concerns,  including  sequestration,  according  to  officials.  Program  officials  noted  that  the 
most  significant  risk  is  integration  with  the  existing  systems  being  leveraged,  such  as  the  AIM  9  Class 
Interceptor  and  the  Sentinel  radar.  Successful  integration  is  necessary  to  avoid  increased  costs  or  overall 
failure  to  meet  requirements.  According  to  program  officials,  the  program  is  reusing  existing  technologies  to 
promote  affordability  and  using  open  systems  architecture  for  the  launcher  to  enable  it  to  work  with  a  variety 
of  current  and  future  missiles.  Blocks  2  and  3  are  scheduled  to  begin  development  in  fiscal  years  2019  and 
2027,  respectively. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $2.5  billion 

Research  and  development:  $510  million 

Procurement:  $2.0  billion 

Quantity:  5  (development),  344  (procurement) 

Next  Major  Program  Event:  System  requirement  review,  January  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 


MML  OVERVIEW 


30  KW  Generator  |TRL  6) 
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Gear  Bearing 
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|TRl  3) 


Source:  U.S.  Army. 
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Common  Name:  JAGM 


Joint  Air-to-Ground  Missile  (JAGM) 

The  Joint  Air-to-Ground  missile  is  an  Army-led 
program  with  joint  requirements  from  the  Navy  and 
Marine  Corps.  The  missile  is  designed  to  be  air- 
launched  from  helicopters  and  unmanned  aircraft 
systems  to  target  tanks;  light  armored  vehicles; 
missile  launchers;  command,  control,  and 
communications  vehicles;  bunkers;  and  buildings.  It 
is  intended  to  provide  line-of-sight  and  beyond  line- 
of-sight  capabilities  and  deploy  in  a  fire-and-forget 
mode  or  a  precision  attack  mode.  JAGM  will 
replace  all  variants  of  HELLFIRE  missiles. 


Source:  U.S.  Army. 

Current  Status 

In  early  2012,  the  Army  restructured  JAGM  extending  technology  development  by  more  than  2  years  to 
explore  evolutionary  alternatives  to  the  acquisition  strategy,  refine  requirements  and  explore  a  more 
affordable  solution.  In  August  and  November  2012,  the  Army  awarded  letter  contracts  to  Lockheed  Martin 
and  Raytheon  for  this  extended  technology  development.  Program  officials  stated  that  these  efforts  focus  on 
the  guidance  section,  the  key  technology  that  provides  the  capabilities  required  by  the  Army,  Marine  Corps, 
and  Navy.  Preliminary  design  reviews  of  each  contractor's  design  were  held  in  February  and  April  2013, 
respectively. 

The  Assistant  Secretary  of  the  Army  for  Acquisition,  Technology,  and  Logistics  conducted  a  program  status 
review  informed  by  the  preliminary  design  reviews  to  assess  the  program's  cost,  schedule,  and  performance 
risk.  Based  on  funding  shortfalls  of  $36  million  and  findings  of  the  risk  assessment,  in  August  2013,  the 
Assistant  Secretary  of  the  Army  for  Acquisition,  Technology,  and  Logistics  directed  the  JAGM  program  to 
execute  the  continued  technology  development  phase  with  a  single  contractor  in  order  to  meet  technology 
development  objectives  on  schedule.  According  to  program  officials,  Raytheon's  technical  performance  was 
not  on  track  to  meet  program  goals  or  Army  requirements.  As  a  result,  the  government  did  not  award  a 
follow-on  contract  to  extend  performance  after  April  2013.  Lockheed  Martin  will  continue  with  technology 
development  under  their  existing  contract.  Upon  completion  of  all  objectives  and  technology  development 
exit  criteria,  approval  of  the  acquisition  strategy,  and  completion  of  the  source  selection  evaluation  board, 
the  Army  is  planning  for  a  system  development  decision  in  fiscal  year  2015.  The  acquisition  strategy  will 
allow  for  a  competitive  contract  award  for  system  development. 

Estimated  Total  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  Program:  TBD 

Research  and  development  (fiscal  years  2008-2018):  $656.4  million 
Procurement  (fiscal  years  2008  to  2018):  $116.8  million 
Quantity:  167 

Next  Major  Program  Event:  Subcomponent  critical  design  review,  January  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 
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Common  Name:  JLENS 


Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System  (JLENS) 

The  Army's  JLENS  is  designed  to  provide  elevated, 
persistent,  over-the-horizon  surveillance  and  fire 
control  quality  data  enabling  protection  of  U.S.  and 
coalition  forces  and  critical  assets.  A  JLENS  orbit 
consists  of  two  systems:  a  fire  control  radar  system 
and  a  surveillance  radar  system.  Each  system  is 
comprised  of  a  74-meter  tethered  aerostat,  a  mobile 
mooring  station,  communication  and  processing 
stations,  and  ground  support  equipment.  The 
program  was  restructured  following  a  Nunn- 
McCurdy  unit-cost  breach  of  the  critical  threshold. 


Current  Status 

In  August  201 3,  the  Linder  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the 
program's  revised  acquisition  program  baseline,  re-designated  the  program's  acquisition  category  and 
delegated  milestone  decision  authority  to  the  Secretary  of  the  Army.  The  JLENS  program  satisfied 
developmental  testing  and  evaluation  requirements  and  is  proceeding  with  plans  to  execute  a  3-year 
operational  combatant  command  exercise.  The  exercise  is  expected  to  demonstrate  JLENS  capabilities  for 
homeland  defense  and  inform  a  future  decision  for  enduring  operational  employment. 

The  program  reported  that  recent  operational  test  events  at  the  Utah  Test  and  Training  Range  and  at  the 
White  Sands  Missile  Range  demonstrate  that  JLENS  performs  as  expected.  Phase  two  of  the  second  early 
user  test — which  assessed  system  performance  and  reliability  and  allowed  for  soldier  training  on  system 
configuration — was  successfully  completed  in  June  2013,  and,  according  to  program  officials,  eliminated  the 
need  for  the  third  series  of  developmental  tests.  Integrated  fire  control  testing  occurred  in  July  2013  and 
demonstrated  JLENS  capabilities  in  acquiring,  tracking  and  providing  sensor  track  data.  The  program  will 
need  to  complete  "friend  or  foe"  identification  system  and  communication  link  certification  events  before  it 
can  participate  in  the  exercise.  Site  construction  for  the  deployment  of  the  exercise  will  begin  at  Aberdeen 
Proving  Ground  after  the  February  2014  construction  contract  award.  The  construction  will  involve 
completing  aerostat  pads,  roads,  operation  and  support  facilities,  and  infrastructure.  The  initial  system  is 
expected  to  arrive  at  the  exercise  site  location  in  June  2014  and  initial  capability  delivery  is  expected  for  the 
surveillance  radar  in  September  2014  and  the  fire  control  radar  system  in  December  2014. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $2,782.12  million 
Research  and  development:  $2,741.61  million 
Military  construction:  $40.51  million 
Quantity:  2 

Next  Major  Program  Event:  February  2014  construction  contract  award  for  exercise  site  construction. 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  program  office  provided 
technical  comments,  which  were  incorporated  where  deemed  appropriate. 


Page  138 


GAO-1 4-340SP  Assessments  of  Selected  Weapon  Programs 


Common  Name:  MGUE 


Military  GPS  User  Equipment  (MGUE),  Increment  1 

The  Air  Force's  MGUE  program  plans  to  develop 
GPS  receivers  compatible  with  the  military's  next- 
generation  GPS  signal,  Military-Code.  The 
modernized  receivers  are  to  provide  U.S.  forces 
with  enhanced  position,  navigation,  and  time 
capabilities,  while  improving  resistance  to  existing 
and  emerging  threats,  such  as  jamming.  The 
program  is  to  be  completed  in  two  increments. 

Increment  1,  assessed  here,  leverages 
technologies  from  the  Modernized  User  Equipment 
(MUE)  program  to  develop  two  variants  and  begin 
development  of  the  Common  GPS  Module. 


Current  Status 

In  April  2012,  the  Air  Force  initiated  technology  development  for  Increment  1  of  MGUE.  According  to  the  Air 
Force,  all  five  critical  technologies  identified  by  the  Office  of  the  Secretary  of  Defense — military-code 
acquisition  engine,  military-code  cryptography,  selective  availability/anti-spoofing  module  cryptography,  anti¬ 
spoofing,  and  anti-tamper-are  currently  nearing  maturity.  The  Air  Force  plans  to  develop  two  variants — one 
each  for  ground  and  aviation,  as  Navy  officials  believe  the  aviation  variant  can  support  maritime  needs.  The 
current  approved  Increment  1  program  leverages  technologies  from  the  Modernized  User  Equipment 
program  to  develop  two  variants  and  begin  development  of  the  Common  GPS  Module.  Increment  2  is  not 
yet  approved. 

The  program's  preliminary  design  review  has  been  delayed  5  months,  to  June  2014.  According  to  officials, 
the  delay  was  caused  by  two  factors:  sequestration  of  fiscal  year  201 3  MGUE  funding,  which  forced  the 
program  to  limit  contractor  activity,  and  a  new  initiative  to  reduce  complexity  within  the  cryptography  required 
to  encrypt  and  decrypt  communications.  In  last  year's  assessment,  the  program  reported  two  main  risk 
areas,  ensuring  more  than  one  contractor  meets  design  requirements  for  the  application  specific  integrated 
circuit  (ASIC) — a  vital  component  leveraged  from  MUE — and  software  development.  According  to  Air  Force 
officials,  progress  in  manufacturing  and  testing  the  ASIC  as  well  as  the  complexity  reduction  effort  has 
largely  addressed  these  risk  areas.  However,  the  program  identified  a  new  risk  in  obtaining  security 
certification  to  field  GPS  receivers  in  unclassified  environments.  According  to  officials,  however,  neither  the 
delay  in  the  preliminary  design  review  nor  the  risk  areas  is  expected  to  delay  the  program's  entry  into  system 
development,  scheduled  for  November  2014. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development:  $1,658.0  million 
Procurement:  TBD 

Quantity:  124  (development),  550  (production) 

Next  Major  Program  Event:  Preliminary  design  review,  April  2014 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  the  Air  Force  affirmed  that  it 
continues  to  strive  to  field  military-code  GPS  receivers  as  quickly  as  possible.  The  service  has  leveraged 
previous  technology  and  industry  experience  to  mitigate  risk  and  reduce  cost,  and  seeks  additional 
opportunities  to  streamline  the  MGUE  acquisition.  The  program  office  also  provided  technical  comments, 
which  have  been  incorporated  where  appropriate. 
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Common  Name:  NGJ 


Next  Generation  Jammer  (NGJ) 

The  Navy's  Next  Generation  Jammer  (NGJ)  is  being  - 

developed  as  an  external  jamming  pod  system  fitted 
on  carrier-based  EA-18G  Growler  aircraft.  It  will 
replace  the  ALQ-99  jamming  pod  system  and 
provide  enhanced  airborne  electronic  attack 
capabilities  to  disrupt  and  degrade  enemy  air 
defense  and  ground  communication  systems.  The 
Navy  plans  to  field  capabilities  in  three  increments 
for  different  radio  frequency  ranges  beginning  with 
Increment  1  (mid-band)  in  2020  with  Increments  2 
and  3  (low-  and  high-band)  to  follow.  We  assessed 
Increment  1. 


Source:  U.S.  Navy. 

Current  Status 

In  July  2013,  DOD  approved  NGJ  Increment  1's  entry  into  technology  development;  however,  work  was 
delayed  when  GAO  sustained  a  protest  related  to  the  technology  development  contract  award.  Prior  to 
technology  development,  the  Navy  completed  a  33-month  technology  maturation  phase,  during  which  the 
Navy  selected  four  contractors  to  prototype  key  technologies  and  subsystems  related  to  their  proposed  NGJ 
Increment  1  concept.  According  to  program  officials,  each  contractor  chose  the  technologies  to  prototype 
during  technology  maturation,  but  they  generally  aligned  with  five  critical  technology  areas — power 
generation  and  distribution,  exciters,  beam  formers,  amplifiers,  and  apertures.  DOD  believes  that  these 
prototyping  activities  meet  the  competitive  prototyping  requirements  of  the  Weapon  Systems  Acquisition 
Reform  Act  of  2009,  which  require  competitive  prototyping  for  MDAPs  before  system  development  unless  a 
waiver  is  granted.  The  program  office  stated  it  is  also  planning  to  use  open  systems  architecture  to  enhance 
future  competitions  and  to  provide  flexibility  for  upgrades. 

Following  technology  maturation,  the  Navy  awarded  a  single  $280  million  contract  to  Raytheon  for 
technology  development  with  plans  to  award  a  sole-source  follow-on  contract  to  complete  system 
development  for  Increment  1.  In  October  2011,  the  Navy  stated  that  it  expected  to  save  about  $641  million 
by  using  only  one  contractor  for  technology  development.  In  November  2013,  GAO  sustained  portions  of 
BAE  Systems'  protest  of  the  contract  award  and  recommended  that  the  Navy  document  a  reevaluation  of 
the  proposals  and  a  cost/technical  tradeoff  analysis.  In  January  2014,  the  Navy  reaffirmed  Raytheon's 
contract  award  and  resumed  NGJ's  technology  development  efforts  after  a  6-month  delay,  but  faces  a  $100 
million  funding  reduction. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $6,336.3  million 

Research  and  development:  $2,894.6  million 

Procurement:  $3,442.7  million 

Quantity:  9  (development),  114  (procurement) 

Next  Major  Program  Event:  System  development  start,  second  quarter  fiscal  year  2016 

Program  Office  Comments:  The  NGJ  program  office  provided  technical  comments,  which  were 
incorporated  as  appropriate.  The  Navy  is  reviewing  its  schedule  to  identify  actions  to  maintain  its  fiscal  year 
2020  fielding  date  for  NGJ  Increment  1. 
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Common  Name:  OR 


Oft/o-Class  Replacement  (OR) 

The  Navy's  Ohio- class  Replacement  (OR)  will 
replace  the  current  fleet  of  Ohio- class  ballistic 
missile  submarines  (SSBNs)  as  they  begin  to  retire 
in  2027.  The  Navy  began  technology  development 
in  January  2011  in  order  to  avoid  a  gap  in  sea  based 
nuclear  deterrence  between  the  Ohio- class's 
retirement  and  the  production  of  a  replacement.  The 
Navy  is  working  with  the  United  Kingdom  to  develop 
a  common  missile  compartment  for  use  on  OR  and 
the  United  Kingdom's  replacement  SSBN.  OR  will 
initially  carry  the  Trident  II  D5LE  missile. 


Current  Status 

Navy  officials  stated  that  they  are  pursuing  design  for  affordability  initiatives  and  investigating  various 
contracting  and  acquisition  scenarios  to  reduce  average  follow-on  ship  procurement  costs  from  an 
estimated  $5.6  to  $4.9  billion  (in  fiscal  year  2010  dollars). The  program  intends  to  maximize  economic  order 
quantity  benefits  by  leveraging  the  Virginia- class  program  and  the  elements  common  with  the  United 
Kingdom's  SSBN.  Program  officials  stated  the  OR  and  Virginia- class  programs  are  aligned  to  support 
various  contracting  scenarios,  including  procuring  the  lead  OR  concurrently  in  a  multi-year  procurement  with 
the  Block  V  Virginia- class  contract.  The  Navy  would  develop  a  legislative  proposal  in  2017  to  support  this 
approach.  The  lead  ship  would  likely  not  meet  multi-year  procurement  criteria  that  require  a  stable  design 
that  has  typically  completed  initial  operational  test  and  evaluation.  The  Navy  has  set  initial  configurations  for 
areas  including  the  torpedo  room,  bow,  and  stern.  In  2014,  the  program  expects  to  complete  initial 
specifications,  set  ship  length — a  major  milestone — and  start  detailed  system  descriptions  and 
arrangements.  The  contractor  plans  to  build  a  notional  submarine  section  to  validate  the  fidelity  of  its  new 
design  tool.  The  program  plans  to  have  83  percent  of  design  disclosures  (including  drawings,  and  material 
procurement  and  construction  planning  information)  and  100  percent  of  arrangements  and  the  three- 
dimensional  product  model  completed  prior  to  the  start  of  lead  ship  construction  scheduled  for  fiscal  year 
2021.  Program  officials  stated  that  unmitigated,  a  sequestration  of  2014  funds  would  result  in  up  to  a  one 
year  delay  to  the  program  (potentially  creating  a  gap  in  sea-based  nuclear  deterrence),  but  that  they  will 
attempt  to  mitigate  any  delays  to  the  fullest  extent  allowed. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  $95,103.2  million 
Research  and  development:  $11,718.2  million 
Procurement:  $83,385.0  million 
Quantity:  12 

Next  Major  Program  Event:  System  requirements  review,  fiscal  year  2014 

Program  Office  Comments:  According  to  the  program  office,  the  OR  program  multi-year-procurement 
decision  has  not  been  made  at  this  time.  The  OR  program  intends  to  maximize  savings  from  cross-class 
contracting  with  the  Virginia- class  and  the  United  Kingdom's  Successor- class  to  take  advantage  of  benefits 
from  economic  order  quantity.  The  OR  program  is  currently  conducting  analysis  on  multi-year-procurement 
and  other  contracting  scenarios. 


Source:  ©  2012  General  Dynamics  Electric  Boat. 
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Common  Name:  PAR 


Presidential  Aircraft  Recapitalization  (PAR) 

The  Air  Force's  Presidential  Aircraft  Recapitalization 
(PAR)  program  is  intended  to  recapitalize  the  VC- 
25A  system  and  support  the  United  States 
President  as  Head  of  State,  Chief  Executive,  and 
Commander  in  Chief.  PAR'S  principal  mission  is  to 
provide  the  President  and  his  staff  air  transportation 
with  the  same  level  of  security  and  communications 
capability  available  at  the  White  House. 


Current  Status 

PAR  program  acquisition  strategy  development  is  ongoing  with  milestone  decision  authority  approval 
expected  in  the  fourth  quarter  of  fiscal  year  2015.  According  to  program  officials,  the  program  intends  to 
acquire  a  large,  commercial-derivative  aircraft  and  modify  it  to  meet  desired  capabilities.  The  program 
intends  to  seek  a  waiver  from  the  requirement  to  conduct  competitive  prototyping  prior  to  entry  into 
engineering  and  manufacturing  development.  To  ensure  some  form  of  competition,  the  program  plans  to  use 
open  architecture  for  future  upgrades. 

Since  fiscal  year  2012,  the  PAR  program  has  conducted  initial  planning  activities  including  assessing  risk 
and  systems  engineering  as  well  as  analysis  related  to  requirements,  sustainment,  and  technology  and 
manufacturing  maturity.  Risk  reduction  studies  are  planned  to  continue  through  fiscal  year  2014. 

The  PAR  system  development  start  and  contract  award  are  expected  in  the  fourth  quarter  of  fiscal  year 
2015.  Program  officials  reported  that  at  present,  the  Air  Force  is  still  determining  the  number  of  aircraft 
needed  to  replace  the  current  VC-25A  fleet  and  no  decision  has  been  made  on  PAR  fleet  size.  The  first 
aircraft  will  be  used  to  support  research,  development,  test,  and  evaluation.  Once  the  development  is 
completed,  the  aircraft  will  be  delivered  as  a  fully  capable  aircraft  to  support  presidential  missions  in  fiscal 
year  2023.  Additional  deliveries  will  commence  starting  in  fiscal  year  2025.  According  to  program  officials, 
requirements  are  being  finalized  with  Air  Mobility  Command  but  at  the  time  of  our  review,  critical 
technologies  have  not  yet  been  identified.  Officials  reported  that  program  affordability  is  under  discussion, 
but  no  targets  for  the  final  program  cost  have  been  determined  yet. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development  (fiscal  years  2010  through  2018):  $1,075.9  million 
Procurement:  TBD 
Quantity:  TBD 

Next  Major  Program  Event:  Milestone  decision  authority  approval  of  acquisition  strategy,  fourth  quarter 
fiscal  year  2015. 

Program  Office  Comments:  Technical  comments  were  incorporated  as  appropriate. 
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Common  Name:  SBIRS  High 


Space  Based  Infrared  Satellite  System  (SBIRS  High) 

The  Air  Force's  SBIRS  High  satellite  system  is 
being  developed  to  replace  the  Defense  Support 
Program  and  perform  a  range  of  missile  warning, 
missile  defense,  technical  intelligence,  and 
battlespace  awareness  missions.  SBIRS  High  will 
consist  of  four  satellites  in  geosynchronous  earth 
orbit  (GEO),  two  sensors  on  host  satellites  in  highly 
elliptical  orbit  (HEO),  two  replenishment  satellites 
and  sensors,  and  fixed  and  mobile  ground  stations. 

We  assessed  the  space  segment  and  made 
observations  about  the  ground  segment. 

Source:  ©  2007  Lockheed  Martin. 

Current  Status 

Both  HEO  sensors  and  the  first  two  GEO  satellites  have  been  launched.  One  of  two  replenishment  sensors 
was  delivered  to  the  host  this  year  for  integration,  and  the  third  and  fourth  satellites  are  currently  in 
production.  In  October  2013,  the  first  GEO  satellite  was  accepted  as  operational,  with  previously  identified 
issues  resolved.  This  means  that  the  data  returned  from  the  satellite  is  reliable.  The  second  satellite  was 
accepted  for  operations  in  November  2013.  Each  GEO  satellite  carries  a  scanning  sensor  and  a  staring 
sensor,  which  provide  different  data  to  meet  program  missions.  The  program  launched  the  first  satellite 
without  event  recovery  software  intended  to  re-establish  ground  control  of  the  satellite  in  the  event  of  an 
unforeseen  failure,  so  that  other  software  issues  could  be  addressed.  Given  successful  recovery  software 
testing  on  the  second  GEO  satellite,  the  Air  Force  plans  to  upload  the  software  to  the  first  satellite  in  the 
fourth  quarter  of  2014. 

The  Block  10  software  ground  system  increment  is  expected  to  be  delivered  in  March  2016,  and  is  intended 
to  facilitate  processing  of  integrated  data  from  the  Defense  Support  Program  satellites,  HEO  sensors  and 
GEO  satellites  now  on-orbit.  According  to  the  program,  this  software  delivery  will  also  provide  the  capability 
for  fully-tuned  starer  sensor  data,  which  means  that  the  data  returned  from  the  satellites  will  be  cleared  of 
background  noise,  such  as  irrelevant  light  sources.  This  capability  was  previously  planned  for  inclusion  in  a 
subsequent  Block  20  software  delivery  which  is  expected  to  achieve  certification  in  June  201 8.  The  program 
plans  to  fully  meet  operational  requirements  in  2019  once  it  has  established  the  full  on-orbit  constellation  of 
HEO  sensors,  four  GEO  satellites,  completion  of  Blocks  10  and  20,  and  delivery  of  its  mobile  ground  assets. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  Program:  $18,886.4  million 
Research  &  development:  $11,805.5  million 
Procurement:  $6,817.2  million 
Quantity:  6 

Next  Major  Program  Event:  Third  GEO  satellite  available  for  delivery  December  2015 

Program  Office  Comments:  In  commenting  on  a  draft  of  this  assessment,  program  officials  noted  that  the 
first  and  second  satellites  are  operationally  accepted  by  US  Strategic  Command.  The  third  is  expected  early, 
but  will  need  to  be  stored.  The  ground  system,  and  fourth  sensor  and  satellite  are  on-track.  The  fifth  and 
sixth  satellite  production  contract  is  expected  fiscal  year  2014. 
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Common  Name:  3DELRR 


Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

The  Air  Force's  3DELRR  is  being  developed  as  a 
long-range,  ground-based  sensor  for  detecting, 
identifying,  tracking,  and  reporting  aircraft  and 
missiles  for  the  Joint  Forces  Air  Component 
Commander.  It  is  intended  to  provide  real-time  data 
and  support  a  range  of  expeditionary  operations  in 
all  types  of  weather  and  terrain.  It  is  being  acquired 
to  replace  the  Air  Force's  AN/TPS-75  radar 
systems. 


Current  Status 

The  3DELRR  program  entered  technology  development  in  May  2009,  and  these  efforts  included  capability 
demonstrations  of  three  competing  prototypes.  The  program  is  scheduled  to  enter  system  development  in 
June  2014  after  it  completes  the  source  selection  process  for  its  development  and  initial  production  contract. 
The  acquisition  strategy  for  the  program  has  changed  in  the  last  year.  The  program  previously  planned  to 
award  the  system  development  and  low-rate  initial  production  contract  using  full  and  open  competition. 
However,  when  the  Air  Force  released  the  request  for  proposals  for  this  contract  in  November  2013,  the 
program  limited  the  competition  to  the  three  contractors  that  had  developed  full-scale  prototypes  and 
completed  a  preliminary  design  review — Lockheed  Martin,  Northrop  Grumman,  and  Raytheon.  According  to 
program  officials,  including  other  contractors  would  require  more  development  funding  to  reach  maturity  and 
these  costs  would  not  likely  be  recouped  through  full  and  open  competition.  However,  the  program  plans  to 
take  other  steps,  such  as  obtaining  data  rights,  to  maintain  the  ability  to  introduce  competition  for 
subsystems  and  upgrades  later.  The  maximum  total  ceiling  price  for  the  planned  contract  is  $534  million, 
which  includes  an  option  for  low-rate  initial  production. 

The  3DELRR  program  will  enter  system  development  with  its  critical  technologies  nearing  maturity.  The 
program  office  reported  that  3DELRR  successfully  demonstrated  its  eight  critical  technologies  and  its 
manufacturing  processes  in  a  relevant  environment  during  its  technology  development  efforts.  According  to 
the  program  office,  the  greatest  risk  to  program  execution  is  budget  uncertainty. 

Estimated  Total  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  Program:  $1,531.4  million 
Research  and  development:  $543.4  million 
Procurement:  $988.1  million 
Quantities:  35 

Next  Major  Program  Event:  System  development  start,  June  2014 

Program  Office  Comments:  The  program  office  concurred  with  this  assessment. 
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Common  Name:  UCLASS 


Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike  (UCLASS)  System 

The  Navy's  UCLASS  system  is  expected  to  address 
a  gap  in  persistent  sea-based  intelligence, 
surveillance,  and  reconnaissance  (ISR)  with 
precision  strike  capabilities.  The  system  is  made  up 
of  three  key  segments  an  unmanned  aerial  vehicle, 
aircraft  carrier  modifications,  and  a  control  system. 

Working  together  these  segments  will  provide 
aircraft  carriers  with  additional  ISR  as  well  as 
targeting  and  strike  capabilities. 


Current  Status 

In  June  2013,  the  Under  Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics  approved  the 
Navy's  plan  to  invest  an  estimated  $3.7  billion  through  fiscal  year  2020  to  develop,  produce,  and  field  an 
initial  UCLASS  system  with  up  to  24  air  vehicles  and  modify  up  to  four  aircraft  carriers,  to  support  an  early 
operational  capability.  The  Navy  plans  to  manage  UCLASS  as  a  technology  development  program,  and 
does  not  plan  to  hold  a  formal  review  to  enter  system  development  until  after  an  initial  system  is  fielded  in 
fiscal  year  2020.  Normally,  activities  like  those  planned  by  the  Navy  are  commensurate  with  an  acquisition 
program  in  system  development  and  early  production.  Using  this  approach,  the  Navy  is  considering  seeking 
approval  to  bypass  a  formal  system  development  phase  and  proceed  directly  into  production  in  fiscal  year 
2020  but  no  formal  decision  has  been  made.  This  approach  raises  concerns  as  the  Navy  could  develop, 
produce,  and  field  a  system  before  undergoing  the  key  oversight  mechanisms  and  reviews  that  typically 
govern  a  program  in  system  development. 

In  August  2013,  the  Navy  awarded  four  separate  limited  source  firm  fixed-price  contracts  to  develop  designs 
for  the  air  vehicle  segment.  In  fiscal  year  2014,  the  Navy  plans  to  review  those  preliminary  designs,  conduct 
a  full  and  open  competition,  and  award  a  single  contract  to  complete  development  and  deliver  the  air 
vehicles.  UCLASS  is  critically  dependent  on  the  development  and  fielding  of  the  Joint  Precision  Approach 
and  Landing  System  (JPALS),  a  global  positioning  system  that  guides  aircraft  onto  an  aircraft  carrier.  Navy 
officials  expect  UCLASS  to  hold  a  preliminary  design  review — including  the  air  vehicle,  carrier,  and  control 
segments — in  May  2014  based  on  JPALS  test  progress.  However,  the  Navy  still  considers  JPALS  one  of  its 
top  risks  for  UCLASS. 

Estimated  Program  Cost  and  Quantity  (fiscal  year  2014  dollars): 

Total  program:  TBD 

Research  and  development:  $3,700  million 
Procurement:  TBD 
Quantity:  TBD 

Next  Major  Program  Event:  Technology  development  start,  2014 

Program  Office  Comments:  According  to  program  officials,  the  UCLASS  program  continues  to  progress 
toward  conducting  a  milestone  A  defense  acquisition  board  review  in  accordance  with  the  program's 
approved  technology  development  strategy.  A  milestone  B  decision  will  be  sought  and  appropriate 
documentation  submitted  for  review.  The  program  also  provided  technical  comments,  which  were 
incorporated  as  appropriate. 


Source:  U.S.  Navy. 


Page  145 


GAO-1 4-340SP  Assessments  of  Selected  Weapon  Programs 


Agency  Comments 
and  Our  Evaluation 

DOD  provided  written  comments  on  a  draft  of  this  report.  The  comments 
are  reprinted  in  appendix  VII.  We  also  received  technical  comments  from 
DOD,  which  have  been  addressed  in  the  report  as  appropriate. 

In  its  comments,  DOD  stated  that  our  assessment  shows  that  the 
Department's  commitment  to  acquisition  excellence  in  the  "Better  Buying 
Power"  initiatives  continues  to  have  a  positive  impact  on  the  cost, 
schedule,  and  performance  measures  of  the  programs  we  assessed.  As 
we  note  in  the  report,  these  initiatives  seem  to  be  having  a  positive  effect 
at  least  in  the  near  term.  This  is  particularly  true  for  DOD's  implementation 
of  "should-cost"  reviews  which  many  programs  reported  were  responsible 
for  significant  savings.  Our  assessment  also  notes  that  there  are  still 
opportunities  to  find  additional  savings  with  two  other  acquisition  reform 
efforts — affordability  constraints  and  the  promotion  of  competition  before 
and  after  development  start. 

Our  report  also  emphasizes  that  further  implementation  of  knowledge- 
based  best  practices  are  fundamental  to  containing  cost  growth  and 
ensuring  timely  delivery  of  the  capabilities  promised  to  the  warfighter  in  the 
long  term.  These  include  early  systems  engineering  reviews,  mature 
technologies  before  development  start,  and  the  demonstration  of  mature 
designs  through  system-level  prototyping. 

We  are  sending  copies  of  this  report  to  appropriate  congressional 
committees;  the  Secretary  of  Defense;  the  Secretaries  of  the  Army,  Navy, 
and  Air  Force;  and  the  Director  of  the  Office  of  Management  and  Budget. 

In  addition,  the  report  will  be  made  available  at  no  charge  on  the  GAO 

Web  site  at  http://www.gao.gov. 

If  you  or  your  staff  have  any  questions  concerning  this  report,  please 
contact  me  at  (202)  512-4841 .  Contact  points  for  our  offices  of 
Congressional  Relations  and  Public  Affairs  may  be  found  on  the  last  page 
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of  this  report.  Staff  members  making  key  contributions  to  this  report  are 
listed  in  appendix  VIII. 


Michael  J.  Sullivan 

Director,  Acquisition  and  Sourcing  Management 
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List  of  Committees 


The  Honorable  Carl  Levin 
Chairman 

The  Honorable  James  M.  Inhofe 
Ranking  Member 
Committee  on  Armed  Services 
United  States  Senate 

The  Honorable  Richard  J.  Durbin 
Chairman 

The  Honorable  Thad  Cochran 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
United  States  Senate 

The  Honorable  Howard  R  "Buck"  McKeon 
Chairman 

The  Honorable  Adam  Smith 
Ranking  Member 
Committee  on  Armed  Services 
House  of  Representatives 

The  Honorable  Rodney  Frelinghuysen 
Chairman 

The  Honorable  Pete  Visclosky 
Ranking  Member 
Subcommittee  on  Defense 
Committee  on  Appropriations 
House  of  Representatives 
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Appendix  I 

Scope  and  Methodology 


To  develop  our  observations  on  the  overall  changes  in  the  size,  cost,  and 
cycle  time  of  Department  of  Defense’s  portfolio  of  major  defense 
acquisition  programs,  we  obtained  and  analyzed  cost,  quantity,  and 
schedule  data  from  Selected  Acquisition  Reports  (SAR)  and  other 
information  in  the  Defense  Acquisition  Management  Information  Retrieval 
(DAMIR)  Purview  system,  referred  to  as  DAMIR.1  We  converted  all  cost 
information  to  fiscal  year  2014  dollars  using  conversion  factors  from  the 
DOD  Comptroller’s  National  Defense  Budget  Estimates  for  Fiscal  Year 
2014  (table  5-9).  Data  for  the  total  planned  investment  of  major  defense 
acquisition  programs  were  obtained  from  DAMIR,  which  we  aggregated  for 
all  programs  using  fiscal  year  2014  dollars.  Through  discussions  with  DOD 
officials  responsible  for  the  database  and  confirming  selected  data  with 
program  offices,  we  determined  that  the  SAR  data  and  the  information 
retrieved  from  DAMIR  were  sufficiently  reliable  for  our  purposes.  In 
general,  we  refer  to  the  80  major  defense  acquisition  programs  with  SARs 
dated  December  2012  as  DOD’s  2013  or  current  portfolio  and  use  a 
similar  convention  for  prior  year  portfolios.  We  compared  the  programs 
that  issued  SARs  in  December  2012  with  the  list  of  programs  that  had 
issued  SARs  in  December  2011  (2012  portfolio)  to  identify  the  programs 
that  exited  and  entered  the  current  portfolio.  The  Missile  Defense  Agency’s 
Ballistic  Missile  Defense  System  is  excluded  from  all  analyses  as  the 
program  does  not  have  an  integrated  long-term  baseline  which  prevents 
us  from  assessing  the  program’s  cost  progress  or  comparing  it  to  other 
major  defense  acquisition  programs. 

To  determine  the  portfolio  trends  over  the  past  ten  years  we  collected  data 
from  the  annual  December  SARs  for  the  years  2003  (2004  portfolio) 
through  2012  (2013  portfolio).  The  2009  portfolio  is  excluded  because  no 
annual  SARs  were  released  for  the  December  2008  submission  date.  We 
then  analyzed  the  data  to  determine  the  number  of  programs  in  each 
portfolio  year  as  well  as  the  year-by-year  totals  for  research  and 
development,  procurement,  and  other  acquisition  funding  for  each  portfolio 
year  as  well  as  the  total  amount  of  funding  invested  or  remaining  and,  in 
specific  cases  the  amount  due  to  cost  growth.  We  also  used  DAMIR  and 
SAR  data  to  make  a  determination  of  system  type  for  each  program  to 
determine  the  mix  of  system  types  in  each  portfolio  year. 


^AMIR  Purview  is  an  executive  information  system  operated  by  the  Office  of  the  Under 
Secretary  of  Defense  for  Acquisition,  Technology,  and  Logistics/Acquisition  Resources  and 
Analysis. 


Analysis  of  the  Cost 
Performance  of  DOD’s 
Portfolio  of  Major 
Defense  Acquisition 
Programs 
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To  determine  the  cost  and  schedule  changes  on  defense  acquisition 
programs  in  the  current  portfolio  over  the  past  year,  5  years,  and  from 
baseline  estimates,  we  collected  data  from  December  2012,  December 
2011,  and  December  2007  SARs;  acquisition  program  baselines;  and 
program  offices.  For  programs  less  than  a  year  old,  we  calculated  the 
difference  between  the  December  2012  SAR  current  estimate  and  the  first 
full  estimate  to  identify  the  cost  and  schedule  change  over  the  past  year. 
For  programs  less  than  5  years  old,  we  took  a  similar  approach  when 
calculating  the  cost  and  schedule  change  over  the  past  5  years.  We 
retrieved  data  on  research,  development,  test,  and  evaluation; 
procurement;  total  acquisition  cost,  and  schedule  estimates  for  the  80 
programs  in  the  2013  portfolio.2  In  some  cases,  we  divided  four  programs 
into  two  distinct  elements,  because  DOD  reports  performance  data  on 
them  separately.  As  a  result  some  of  our  analysis  reflects  a  total  of  84 
programs  and  sub-elements.  We  analyzed  the  data  to  determine  the 
change  in  research  and  development,  procurement,  and  total  acquisition 
costs  as  well  as  schedule  changes  from  the  first  full  estimate,  generally 
development  start,  to  the  current  estimate  in  the  December  2012  SAR.  For 
the  programs  that  did  not  have  a  development  estimate,  we  compared  the 
current  estimate  to  the  production  estimate.  Also,  for  the  shipbuilding 
programs  that  had  a  planning  estimate,  we  compared  the  current  estimate 
to  the  planning  estimate.  For  programs  that  began  as  non-major  defense 
acquisition  programs,  the  first  full  estimate  we  used  as  a  baseline  may  be 
different  than  the  original  baseline  disclosed  in  DOD  SARs.  We  obtained 
schedule  information  and  calculated  the  cycle  time  from  program  start  to 
initial  operational  capability  and  the  delay  in  obtaining  initial  operational 
capability.  For  programs  in  the  current  portfolio  where  schedule  data  for 
initial  operational  capability  was  not  available  over  the  past  year,  5  years, 
and  since  first  full  estimates  we  used  the  same  methodology  as  used 
when  calculating  cost  change. 

To  determine  whether  programs  experienced  an  increase  or  decreases  in 
buying  power  over  the  past  year,  we  obtained  data  on  program  acquisition 
unit  cost  to  determine  whether  a  program’s  buying  power  had  increased  or 
decreased.  We  reviewed  SARs  for  those  programs  with  changes  in  buying 
power  over  the  past  year  to  measure  the  extent  to  which  changes  in 


2We  refer  to  research,  development,  test,  and  evaluation  costs  as  research  and 
development  or  simply  as  development  costs  in  this  report.  Total  acquisition  cost  includes 
research  and  development  and  procurement  costs  as  well  as  acquisition  related  operation 
and  maintenance  and  system-specific  military  construction  costs. 
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quantity  impacted  procurement  cost  changes.  When  analyzing  buying 
power  changes,  we  also  calculated  the  amount  of  procurement  cost 
growth  attributable  to  quantity  changes,  we  isolated  the  change  in 
procurement  quantities  and  the  prior-year’s  acquisition  procurement  unit 
cost  for  programs  over  the  past  year.  For  those  programs  with  a  change  in 
procurement  quantities,  we  calculated  the  amount  attributable  to  quantity 
changes  as  the  change  in  quantity  multiplied  by  the  average  procurement 
unit  cost  for  the  program  a  year  ago.  The  resulting  dollar  amount  is 
considered  a  change  due  solely  to  shifts  in  the  number  of  units  procured 
and  may  overestimate  the  amount  of  change  expected  when  quantities 
increase  and  underestimate  the  expected  change  when  quantities 
decrease  as  it  does  not  account  for  other  effects  of  quantity  changes  on 
procurement  such  as  gain  or  loss  of  learning  in  production  that  could  result 
in  changes  to  unit  cost  over  time  or  the  use  or  absence  of  economic  orders 
of  material.  However,  these  changes  are  accounted  for  as  part  of  the 
change  in  cost  not  due  to  quantities. 

To  evaluate  program  performance  against  high-risk  criteria  discussed  by 
DOD,  the  Office  of  Management  and  Budget  (OMB),  and  GAO,  we 
calculated  how  many  programs  had  less  than  a  2  percent  increase  in  total 
acquisition  cost  over  the  past  year,  less  than  a  10  percent  increase  over 
the  past  5  years,  and  less  than  a  15  percent  increase  from  first  full 
estimates  using  data  from  December  2012,  December  2011,  and 
December  2007  SARs;  acquisition  program  baselines;  and  program 
offices.  For  programs  that  began  as  non-major  defense  acquisition 
programs,  the  first  full  estimate  we  used  as  a  baseline  may  be  different 
than  the  original  baseline  contained  in  DOD  SARs.  We  also  compared  the 
performance  of  the  2013  portfolio  in  each  high-risk  category  with  the 
performance  of  the  2012  and  2011  portfolios  we  reported  on  in  prior  years 
to  identify  any  positive  or  negative  changes.  For  programs  with  multiple 
sub-programs  presented  in  the  SARs  we  calculated  the  net  effect  of  the 
sub-programs  to  reach  an  aggregate  program  result. 

To  discern  the  cost  and  schedule  performance  of  the  various  system  types 
represented  in  the  current  portfolio,  we  first  determined  each  program’s 
system  type  using  DAMIR  reported  information  on  commodity  type,  the 
program’s  mission  and  description  summary  from  the  December  2012 
SAR,  and  GAO  analyst  judgment.  The  programs  were  then  grouped  by 
system  type  and  cost  and  schedule  change  was  determined  using  the  cost 
and  schedule  changes  calculated  for  the  programs  in  each  system  type. 
Development  cost  change  and  change  in  initial  operating  capability  were 
then  calculated  for  each  group  as  elsewhere  in  this  objective.  For  change 
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in  procurement  unit  cost,  a  weighted  average  calculation  was  used  based 
on  the  total  procurement  cost  for  programs  in  each  system  type. 

To  determine  the  funding  already  invested  and  the  funding  remaining  that 
is  needed  to  complete  the  80  programs  in  the  2013  portfolio  we  used 
funding  stream  data  obtained  from  DAMIR  and  the  December  2012  SARs. 
We  define  funding  invested  as  all  funding  that  has  been  provided  to  the 
programs  in  fiscal  years  2013  and  earlier,  while  funding  remaining  is  all  the 
amounts  that  will  be  provided  in  the  fiscal  year  2014  and  later.  To  show  the 
amount  of  additional  future  funding  needed  due  to  cost  growth  from  first 
full  estimates  to  complete  programs  in  the  current  portfolio,  we  compared 
a  funding  stream  for  each  program  that  corresponded  to  its  first  full 
estimate  at  program  start  against  the  funding  stream  for  the  current 
estimate  in  the  program’s  December  2012  SAR. 


Analysis  of  Selected 
DOD  Programs  Using 
Knowledge-Based 
Criteria 


To  collect  data  from  current  and  future  major  defense  acquisition 
programs — including  cost  and  schedule  estimates,  technology  maturity, 
and  planned  implementation  of  acquisition  reforms — we  distributed  two 
electronic  questionnaires,  one  questionnaire  for  the  38  current  programs 
and  a  slightly  different  questionnaire  for  the  18  future  programs.  Both  of 
the  questionnaires  were  sent  by  e-mail  in  an  attached  Microsoft  Word  form 
that  respondents  could  return  electronically.  We  received  responses  from 
all  of  the  programs  we  assessed  from  August  to  November  2013.  To 
ensure  the  reliability  of  the  data  collected  through  our  questionnaires,  we 
took  a  number  of  steps  to  reduce  measurement  error  and  non-response 
error.  These  steps  included  conducting  three  pretests  for  the  future  major 
defense  acquisition  program  questionnaire  and  three  pretests  for  the 
major  defense  acquisition  program  questionnaire  prior  to  distribution  to 
ensure  that  our  questions  were  clear,  unbiased,  and  consistently 
interpreted;  reviewing  responses  to  identify  obvious  errors  or 
inconsistencies;  conducting  follow-up  to  clarify  responses  when  needed; 
and  verifying  the  accuracy  of  a  sample  of  keypunched  questionnaires.  Our 
pretests  covered  each  branch  of  the  military  to  better  ensure  that  the 
questionnaires  could  be  understood  by  officials  within  each  branch. 


Our  analysis  of  how  well  programs  are  adhering  to  a  knowledge-based 
acquisition  approach  focuses  on  38  major  defense  acquisition  programs 
that  are  mostly  in  development  or  the  early  stages  of  production.  To 
assess  the  knowledge  attained  by  key  decision  points  (system 
development  start  or  detailed  design  contract  award  for  shipbuilding 
programs,  critical  design  review  or  fabrication  start  for  shipbuilding 
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programs,  and  production  start),  we  collected  data  from  program  offices 
about  their  knowledge  at  each  point.  In  particular,  we  focused  on  the 
seven  programs  that  crossed  these  key  acquisition  points  in  2013  and 
evaluated  their  adherence  to  knowledge  based  practices.  We  also  provide 
some  insight  into  how  much  knowledge  is  obtained  at  key  junctures  by 
other  programs  we  assessed  as  well.  We  also  included  observations  on 
the  knowledge  that  18  future  programs  expect  to  obtain  before  starting 
development  as  well  as  how  much  knowledge  14  current  programs  expect 
to  obtain  before  reaching  their  production  start.  We  did  not  validate  the 
data  provided  by  the  program  offices,  but  reviewed  the  data  and 
performed  various  checks  to  determine  that  they  were  reliable  enough  for 
our  purposes.  Where  we  discovered  discrepancies,  we  clarified  the  data 
accordingly. 

The  56  current  and  future  programs  included  in  our  assessment  were  in 
various  stages  of  the  acquisition  cycle,  and  not  all  of  the  programs 
provided  information  on  knowledge  obtained  at  each  point.  Programs  were 
not  included  in  our  assessments  at  key  decision  points  if  relevant  data 
were  not  available.  For  each  decision  point,  we  summarized  knowledge 
attainment  for  the  number  of  programs  with  data  that  had  reached  that 
knowledge  point.  Our  analysis  of  knowledge  attained  at  each  key  point 
includes  factors  that  we  have  previously  identified  as  being  key  to  a 
knowledge-based  acquisition  approach,  including  holding  early  systems 
engineering  reviews,  testing  an  integrated  prototype  prior  to  the  design 
review,  using  a  reliability  growth  curve,  planning  for  manufacturing,  and 
testing  a  production-representative  prototype  prior  to  the  making  a 
production  decision.  Additional  information  on  how  we  collect  these  data  is 
found  in  the  product  knowledge  assessment  section  of  appendix  I.  See 
also  appendix  IV  for  a  list  of  the  practices  that  are  associated  with  a 
knowledge-based  acquisition  approach. 


Analysis  of  Acquisition 
Initiatives  and  Program 
Concurrency 


To  determine  how  DOD  has  begun  to  implement  acquisition  reforms,  we 
obtained  and  analyzed  the  DOD  5000.02  acquisition  instruction  in  place  at 
the  time  of  our  review,  the  Weapon  Systems  Acquisition  Reform  Act  of 
2009,  and  the  September  14,  2010,  Under  Secretary  of  Defense  for 
Acquisition,  Technology,  and  Logistics  memorandum  on  “better  buying 
power”  as  well  as  subsequent  memorandums  clarifying  and  implementing 
that  guidance,  including  the  “Better  Buying  Power  2.0”  memorandum.  In 
November  2013,  DOD  released  an  interim  revision  of  its  5000.02 
acquisition  instruction  with  the  intention  of  incorporating  the  policy 
changes  mandated  by  the  act  and  “Better  Buying  Power”  memos.  As  our 
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data  was  collected  prior  to  the  release  of  this  revision  we  did  not  use  the 
new  instruction  as  criteria  for  our  assessments.  To  develop  our 
observations,  we  analyzed  questionnaire  data  received  from  the  38 
current  and  18  future  major  defense  acquisition  programs  in  our 
assessment  to  determine  the  extent  to  which  specific  acquisition  reform 
issues  have  been  implemented  such  as  establishing  affordability 
constraints,  conducting  “should-cost”  analyses,  using  competition 
throughout  the  acquisition  life  cycle,  and  holding  configuration  steering 
boards. 

To  assess  program  concurrency  we  identified  the  programs — among  those 
we  included  in  our  assessment — with  production  start  dates.  We  used  the 
questionnaire  responses  from  those  programs  to  identify  the  dates  for  the 
start  and  end  of  developmental  testing,  compared  those  dates  to  the 
timing  of  each  program’s  production  decision  and  determined  the  number 
of  months,  if  any,  of  developmental  testing  done  after  production  start.  We 
then  compared  the  number  of  overlapping  months  to  the  total  number  of 
months  of  developmental  testing  for  each  program  and  calculated  the 
percentage  of  developmental  testing  done  concurrent  with  production. 


Individual 
Assessments  of 
Weapon  Programs 


In  total,  this  report  presents  individual  assessments  of  56  weapon 
programs.  A  table  listing  these  programs  is  found  in  appendix  VIII.  Out  of 
these  programs,  37  are  captured  in  a  two-page  format  discussing 
technology,  design,  and  manufacturing  knowledge  obtained  and  other 
program  issues.  Thirty-four  of  these  37  two-page  assessments  are  of 
major  defense  acquisition  programs,  most  of  which  are  in  development  or 
early  production  and  3  assessments  are  of  programs  that  were  projected 
to  become  major  defense  acquisition  programs  during  or  soon  after  our 
review.  The  remaining  19  programs  are  described  in  a  one-page  format 
that  describes  their  current  status.  Those  one-page  assessments  include 
15  future  major  defense  acquisition  programs,  2  major  defense  acquisition 
program  that  are  well  into  production,  and  2  major  defense  acquisition 
programs  that  were  recently  restructured  or  curtailed.  Over  the  past 
several  years,  DOD  has  revised  policies  governing  weapon  system 
acquisitions  and  changed  the  terminology  used  for  major  acquisition 
events.  To  make  DOD’s  acquisition  terminology  more  consistent  across 
the  56  program  assessments,  we  standardized  the  terminology  for  key 
program  events.  For  most  individual  programs  in  our  assessment, 
“development  start”  refers  to  the  initiation  of  an  acquisition  program  as  well 
as  the  start  of  engineering  and  manufacturing  development.  This  generally 
coincides  with  DOD’s  milestone  B.  A  few  programs  in  our  assessment 
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have  a  separate  “program  start”  date,  which  begins  a  pre-system 
development  phase  for  program  definition  and  risk-reduction  activities. 
This  “program  start”  date  generally  coincides  with  DOD’s  former 
terminology  for  milestone  I  or  DOD’s  current  milestone  A.  The  “production 
decision”  generally  refers  to  the  decision  to  enter  the  production  and 
deployment  phase,  typically  with  low-rate  initial  production.  The  “initial 
capability”  refers  to  the  initial  operational  capability — sometimes  called  first 
unit  equipped  or  required  asset  availability.  For  shipbuilding  programs,  the 
schedule  of  key  program  events  in  relation  to  acquisition  milestones  varies 
for  each  program.  Our  work  on  shipbuilding  best  practices  has  identified 
the  detailed  design  contract  award  and  the  start  of  lead  ship  fabrication  as 
the  points  in  the  acquisition  process  roughly  equivalent  to  development 
start  and  design  review  for  other  programs. 

For  each  program  we  assessed  in  a  two-page  format,  we  present  cost, 
schedule,  and  quantity  data  at  the  program’s  first  full  estimate  and  an 
estimate  from  the  latest  SAR  or  the  program  office  reflecting  2013  data 
where  they  were  available.  The  first  full  estimate  is  generally  the  cost 
estimate  established  at  milestone  B — development  start;  however,  for  a 
few  programs  that  did  not  have  such  an  estimate,  we  used  the  estimate  at 
milestone  C — production  start — instead.  For  shipbuilding  programs,  we 
used  their  planning  estimates  if  those  estimates  were  available.  For 
systems  for  which  a  first  full  estimate  was  not  available,  we  only  present 
the  latest  available  estimate  of  cost  and  quantities.  For  the  other  programs 
assessed  in  a  one-page  format,  we  present  the  latest  available  estimate  of 
cost  and  quantity  from  the  program  office. 

For  each  program  we  assessed,  all  cost  information  is  presented  in  fiscal 
year  2014  dollars.  We  converted  cost  information  to  fiscal  year  2014 
dollars  using  conversion  factors  from  the  DOD  Comptroller’s  National 
Defense  Budget  Estimates  for  Fiscal  Year  2014  (table  5-9).  We  have 
depicted  only  the  program’s  main  elements  of  acquisition  cost — research 
and  development  and  procurement.  However,  the  total  program  cost  also 
includes  military  construction  and  acquisition-related  operation  and 
maintenance  costs.  Because  of  rounding  and  these  additional  costs,  in 
some  situations,  total  cost  may  not  match  the  exact  sum  of  the  research 
and  development  and  procurement  costs.  The  program  unit  costs  are 
calculated  by  dividing  the  total  program  cost  by  the  total  quantities 
planned.  In  some  instances,  the  data  were  not  applicable,  and  we 
annotate  this  by  using  the  term  “not  applicable  (NA).”  The  quantities  listed 
refer  to  total  quantities,  including  both  procurement  and  development 
quantities. 
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The  schedule  assessment  for  each  program  is  based  on  acquisition  cycle 
time,  defined  as  the  number  of  months  between  program  start  and  the 
achievement  of  initial  operational  capability  or  an  equivalent  fielding  date. 
In  some  instances  the  data  were  not  yet  available,  and  we  annotate  this  by 
using  the  term  “to  be  determined  (TBD)”  or  “NA.” 

The  information  presented  on  the  “funding  needed  to  complete”  is  from 
fiscal  year  2014  through  completion  and,  unless  otherwise  noted,  draws 
on  information  from  SARs  or  on  data  from  the  program  office.  In  some 
instances,  the  data  were  not  available,  and  we  annotate  this  by  the  term 
“TBD”  or  “NA.”  The  quantities  listed  refer  only  to  procurement  quantities. 
Satellite  programs,  in  particular,  produce  a  large  percentage  of  their  total 
operational  units  as  development  quantities,  which  are  not  included  in  the 
quantity  figure. 

The  intent  of  these  comparisons  is  to  provide  an  aggregate,  or  overall, 
picture  of  a  program’s  history.  These  assessments  represent  the  sum  of 
the  federal  government’s  actions  on  a  program,  not  just  those  of  the 
program  manager  and  the  contractor.  DOD  does  a  number  of  detailed 
analyses  of  changes  that  attempt  to  link  specific  changes  with  triggering 
events  or  causes.  Our  analysis  does  not  attempt  to  make  such  detailed 
distinctions. 

In  this  year’s  assessment  we  also  reviewed  whether  individual 
subcontracting  reports  from  a  program’s  prime  contractor  or  contractors 
were  accepted  on  the  Electronic  Subcontracting  Reporting  System 
(eSRS).  We  reviewed  this  information  for  36  of  the  major  defense 
acquisition  programs  included  in  our  assessment  using  the  contract 
information  reported  in  their  December  2012  Selected  Acquisition  Reports. 
See  appendix  VI  for  a  list  of  the  programs  we  reviewed.  The  contract 
numbers  for  each  program’s  prime  contracts  were  entered  into  the  eSRS 
database  to  determine  whether  the  individual  subcontracting  reports  had 
been  accepted  by  the  government.  While  we  did  not  assess  the  reliability 
of  the  eSRS  database,  we  took  steps  to  ensure  that  the  data  provided  by 
the  database  was  sufficiently  reliable  for  the  purpose  of  our  analysis  by 
interviewing  defense  officials  with  knowledge  of  DOD’s  process  for  eSRS 
and  collecting  data  from  program  offices  on  contract  reporting.  The 
government  uses  individual  subcontracting  reports  on  eSRS  as  one 
method  of  monitoring  small  business  participation,  as  the  report  includes 
goals  for  small  business  subcontracting.  Not  all  prime  contracts  for  major 
defense  acquisition  programs  are  required  to  submit  individual 
subcontracting  reports.  For  example,  some  contractors  report  small 
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business  participation  at  a  corporate  level  as  opposed  to  a  program  level 
and  this  data  is  not  captured  in  the  individual  subcontracting  reports. 


Product  Knowledge 
Data  on  Individual 
Two-Page 
Assessments 


In  our  past  work  examining  weapon  acquisition  issues  and  best  practices 
for  product  development,  we  have  found  that  leading  commercial  firms 
pursue  an  acquisition  approach  that  is  anchored  in  knowledge,  whereby 
high  levels  of  product  knowledge  are  demonstrated  by  critical  points  in  the 
acquisition  process.  On  the  basis  of  this  work,  we  have  identified  three  key 
knowledge  points  during  the  acquisition  cycle — system  development  start, 
critical  design  review,  and  production  start — at  which  programs  need  to 
demonstrate  critical  levels  of  knowledge  to  proceed.  To  assess  the  product 
development  knowledge  of  each  program  at  these  key  points,  we 
reviewed  data-collection  instruments  and  questionnaires  submitted  by 
programs;  however,  not  every  program  had  responses  to  each  element  of 
the  data-collection  instrument  or  questionnaire.  We  also  reviewed 
pertinent  program  documentation  and  discussed  the  information 
presented  on  the  data-collection  instrument  and  questionnaire  with 
program  officials  as  necessary. 

To  assess  a  program’s  readiness  to  enter  system  development,  we 
collected  data  through  the  data-collection  instrument  on  critical 
technologies  and  early  design  reviews.  To  assess  technology  maturity,  we 
asked  program  officials  to  apply  a  tool,  referred  to  as  technology  readiness 
levels  (TRL),  for  our  analysis.  The  National  Aeronautics  and  Space 
Administration  originally  developed  TRLs,  and  the  Army  and  Air  Force 
science  and  technology  research  organizations  use  them  to  determine 
when  technologies  are  ready  to  be  handed  off  from  science  and 
technology  managers  to  product  developers.  TRLs  are  measured  on  a 
scale  from  1  to  9,  beginning  with  paper  studies  of  a  technology’s  feasibility 
and  culminating  with  a  technology  fully  integrated  into  a  completed 
product.  See  appendix  V  for  TRL  definitions.  Our  best-practices  work  has 
shown  that  a  TRL  7 — demonstration  of  a  technology  in  a  realistic 
environment — is  the  level  of  technology  maturity  that  constitutes  a  low  risk 
for  starting  a  product  development  program.3  For  shipbuilding  programs, 
we  have  recommended  that  this  level  of  maturity  be  achieved  by  the 


3GAO,  Best  Practices:  Better  Management  of  Technology  Development  Can  Improve 
Weapon  System  Outcomes ,  GAO/NSIAD-99-162  (Washington,  D.C.:  July  30,  1999);  and 
Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to  Better  Weapon 
System  Outcomes,  GAO-01 -288  (Washington,  D.C.:  Mar.  8,  2001). 
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contract  award  for  detailed  design.4  In  our  assessment,  the  technologies 
that  have  reached  TRL  7,  a  prototype  demonstrated  in  a  realistic 
environment,  are  referred  to  as  mature  or  fully  mature.  Those  technologies 
that  have  reached  TRL  6,  a  prototype  demonstrated  in  a  relevant 
environment,  are  referred  to  as  approaching  or  nearing  maturity.  Satellite 
technologies  that  have  achieved  TRL  6  are  assessed  as  fully  mature  due 
to  the  difficulty  of  demonstrating  maturity  in  a  realistic  environment — 
space.  In  addition,  we  asked  program  officials  to  provide  the  date  of  the 
preliminary  design  review.  We  compared  this  date  to  the  system 
development  start  date. 

In  most  cases,  we  did  not  validate  the  program  offices’  selection  of  critical 
technologies  or  the  determination  of  the  demonstrated  level  of  maturity. 
We  sought  to  clarify  the  TRLs  in  those  cases  where  information  existed 
that  raised  concerns.  If  we  were  to  conduct  a  detailed  review,  we  might 
adjust  the  critical  technologies  assessed,  their  readiness  levels 
demonstrated,  or  both.  It  was  not  always  possible  to  reconstruct  the 
technological  maturity  of  a  weapon  system  at  key  decision  points  after  the 
passage  of  many  years.  Where  practicable,  we  compared  technology 
assessments  provided  by  the  program  office  to  assessments  conducted 
by  officials  from  the  Office  of  the  Assistant  Secretary  of  Defense  for 
Research  and  Engineering. 

To  assess  design  stability,  we  asked  program  officials  to  provide  the 
percentage  of  design  drawings  completed  or  projected  for  completion  by 
the  design  review,  the  production  decision,  and  as  of  our  current 
assessment  in  the  data-collection  instrument.  In  most  cases,  we  did  not 
verify  or  validate  the  percentage  of  engineering  drawings  provided  by  the 
program  office.  We  clarified  the  percentage  of  drawings  completed  in 
those  cases  where  information  that  raised  concerns  existed.  Completed 
drawings  were  defined  as  the  number  of  drawings  released  or  deemed 
releasable  to  manufacturing  that  can  be  considered  the  “build  to” 
drawings.  For  shipbuilding  programs,  we  asked  program  officials  to 
provide  the  percentage  of  the  three-dimensional  product  model  that  had 
been  completed  by  the  start  of  lead  ship  fabrication,  and  as  of  our  current 
assessment.  To  gain  greater  insights  into  design  stability,  we  also  asked 
program  officials  to  provide  the  date  they  planned  to  first  integrate  and  test 
all  key  subsystems  and  components  into  a  system-level  integrated 
prototype.  We  compared  this  date  to  the  date  of  the  design  review.  We  did 


4GAO,  Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate  Commercial 
Shipbuilding  from  Navy  Shipbuilding,  GAO-09-322  (Washington,  D.C.:  May  13,  2009). 
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not  assess  whether  shipbuilding  programs  had  completed  integrated 
prototypes. 

To  assess  production  maturity,  we  asked  program  officials  to  identify  the 
number  of  critical  manufacturing  processes  and,  where  available,  to 
quantify  the  extent  of  statistical  control  achieved  for  those  processes  as  a 
part  of  our  data-collection  instrument.  In  most  cases,  we  did  not  verify  or 
validate  the  information  provided  by  the  program  office.  We  clarified  the 
number  of  critical  manufacturing  processes  and  the  percentage  of 
statistical  process  control  where  information  existed  that  raised  concerns. 
We  used  a  standard  called  the  process  capability  index,  a  process- 
performance  measurement  that  quantifies  how  closely  a  process  is 
running  to  its  specification  limits.  The  index  can  be  translated  into  an 
expected  product  defect  rate,  and  we  have  found  it  to  be  a  best  practice. 
We  also  used  data  provided  by  the  program  offices  on  their  manufacturing 
readiness  levels  (MRL)  for  process  capability  and  control,  a  sub-thread 
tracked  as  part  of  the  manufacturing  readiness  assessment  process 
recommended  by  DOD,  to  determine  production  maturity.  We  assessed 
programs  as  having  mature  manufacturing  processes  if  they  reported  an 
MRL  9  for  that  sub-thread — meaning,  that  manufacturing  processes  are 
stable,  adequately  controlled,  and  capable.  To  gain  further  insights  into 
production  maturity,  we  asked  program  officials  whether  the  program 
planned  to  demonstrate  critical  manufacturing  processes  on  a  pilot 
production  line  before  beginning  low-rate  production.  We  also  asked 
programs  on  what  date  they  planned  to  begin  system-level  developmental 
testing  of  a  fully  configured,  production-  representative  prototype  in  its 
intended  environment.  We  compared  this  date  to  the  production  start  date. 
We  did  not  assess  production  maturity  for  shipbuilding  programs. 

Although  the  knowledge  points  provide  excellent  indicators  of  potential 
risks,  by  themselves  they  do  not  cover  all  elements  of  risk  that  a  program 
encounters  during  development,  such  as  funding  instability.  Our  detailed 
reviews  on  individual  systems  normally  provide  a  more  comprehensive 
assessment  of  risk  elements. 

We  conducted  this  performance  audit  from  June  2013  to  March  2014,  in 
accordance  with  generally  accepted  government  auditing  standards. 
Those  standards  require  that  we  plan  and  perform  the  audit  to  obtain 
sufficient,  appropriate  evidence  to  provide  a  reasonable  basis  for  our 
findings  and  conclusions  based  on  our  audit  objectives.  We  believe  that 
the  evidence  obtained  provides  a  reasonable  basis  for  our  findings  and 
conclusions  based  on  our  audit  objectives. 
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Table  8  contains  the  current  and  first  full  total  acquisition  cost  estimates  (in 
fiscal  year  2013  dollars)  for  each  program  or  element  in  the  Department  of 
Defense’s  (DOD)  2013  major  defense  acquisition  program  portfolio.  For 
each  program  we  show  the  percent  change  in  total  acquisition  cost  from 
the  first  full  estimate,  as  well  as  over  the  past  year  and  5  years. 


Table  8:  Current  Cost  Estimates  and  First  Full  Estimates  for  DOD’s  2013  Portfolio  of  Major  Defense  Acquisition  Programs 


Fiscal  year  2014  dollars  in  millions 


Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Advanced  Extremely  High  Frequency  Satellite  (AEHF) 

$14,559 

$6,700 

117.3% 

-0.9% 

70.2% 

AGM-88E  Advanced  Anti-Radiation  Guided  Missile 
(AGM-88E  AARGM) 

2,062 

1,684 

22.5 

-0.4 

15.1 

AH-64E  Apache  New  Build  (AH-64E  New  Build) 

2,119 

2,492 

-15.0 

2.9 

-15.0 

AH-64E  Apache  Remanufacture  (AH-64E  Remanufacture) 

12,499 

7,617 

64.1 

12.9 

50.2 

AIM-120  Advanced  Medium  Range  Air-to-Air  Missile  (AMRAAM) 

24,052 

11,497 

109.2 

-0.9 

23.3 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

3,793 

4,202 

-9.7 

-7.0 

-9.7 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System 
(AMF  JTRS) 

3,583 

8,576 

-58.2 

-12.9 

-58.2 

Airborne  Warning  and  Control  System  Block  40/45  Upgrade 
(AWACS  Blk  40/45  Upgrade) 

2,865 

2,936 

-2.4 

-2.4 

-2.4 

B-2  Extremely  High  Frequency  SATCOM  and  Computer  Increment  1 
(B-2  EHF  Incl ) 

594 

747 

-20.5 

-2.5 

-17.5 

B61  Mod  12  Life  Extension  Program  Tailkit  Assembly 
(B61  Mod  12  LEPTKA) 

1,369 

1,375 

-0.5 

-0.5 

-0.5 

C-130J  Hercules  Transport  Aircraft  (C-130J) 

16,455 

998 

1548.6 

-1.9 

22.9 

C-5  Reliability  Enhancement  and  Re-engining  Program  (C-5  RERP) 

7,661 

11,469 

-33.2 

-1.1 

-30.7 

CH-47F  Improved  Cargo  Helicopter  (CH-47F) 

15,000 

3,386 

343.0 

0.3 

8.7 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

24,806 

17,414 

42.5 

4.7 

42.0 

Chemical  Demilitarization-Assembled  Chemical  Weapons 

Alternatives  (Chem  Demil-ACWA) 

10,496 

2,779 

277.7 

-0.7 

34.5 

Cooperative  Engagement  Capability  (CEC) 

5,491 

3,096 

77.4 

-1.5 

2.8 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

36,008 

37,415 

-3.8 

-0.8 

10.5 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

22,000 

36,602 

-39.9 

0.2 

-26.8 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

107,831 

15,974 

575.0 

2.3 

30.5 
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Fiscal  year  2014  dollars  in  millions 


Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

E-2D  Advanced  Hawkeye  Aircraft  (E-2D  AHE) 

19,899 

15,516 

28.2 

-2.3 

17.1 

EA-18G  Growler  Aircraft  (EA-18G) 

13,681 

9,440 

44.9 

16.1 

46.5 

Evolved  Expendable  Launch  Vehicle  (EELV) 

63,924 

18,276 

249.8 

78.4 

78.8 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

1,882 

5,024 

-62.5 

1.0 

-25.5 

F/A-18E/F  Super  Hornet  Aircraft  (F/A-18E/F) 

59,674 

85,964 

-30.6 

-1.7 

5.8 

F-35  Joint  Strike  Fighter  Aircraft  (F-35) 

332,320 

224,775 

47.8 

-3.4 

24.7 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

4,618 

3,354 

37.7 

-1.4 

22.8 

Family  of  Medium  Tactical  Vehicles  (FMTV) 

19,010 

10,987 

73.0 

-0.4 

-15.7 

Global  Broadcast  Service  (GBS) 

1,276 

606 

110.6 

2.8 

30.8 

Global  Positioning  System  III  (GPS  III) 

4,376 

4,145 

5.6 

1.0 

5.6 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

2,298 

1,542 

49.1 

-26.0 

49.1 

Guided  Multiple  Launch  Rocket  System/Guided  Multiple  Launch 
Rocket  System  Alternative  Warhead  (GMLRS/GMLRS  AW) 

6,520 

1,860 

250.6 

2.9 

10.0 

H-1  Upgrades  (4BW/4BN)  (H-1  Upgrades) 

12,935 

3,814 

239.1 

-1.9 

40.3 

HC/MC-130  Recapitalization  Aircraft  (HC/MC-130  Recap) 

13,788 

8,797 

56.7 

2.9 

56.7 

Integrated  Air  and  Missile  Defense  (IAMD) 

5,804 

5,289 

9.7 

-4.2 

9.7 

Integrated  Defensive  Electronic  Countermeasures  (IDECM) 

2,582 

2,290 

12.7 

2.8 

16.4 

IDECM  Blocks  2/3 

1,720 

1,560 

10.3 

5.5 

10.3 

IDECM  Block  4 

862 

730 

18.0 

-2.1 

18.0 

Joint  Air-to-Surface  Standoff  Missile  -  JASSM  and  JASSM-Extended 
Range  (JASSM) 

7,223 

2,438 

196.3 

-1.2 

16.1 

Joint  Direct  Attack  Munition  (JDAM) 

7,448 

3,596 

107.1 

2.9 

18.2 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor 
System  (JLENS) 

2,782 

7,011 

-60.3 

0.6 

-61.9 

Joint  Light  Tactical  Vehicle  (JLTV) 

23,651 

23,700 

-0.2 

-0.2 

-0.2 

Joint  Precision  Approach  and  Landing  System  Increment  1A 
(JPALS  Inc  1A) 

1,124 

1,065 

5.6 

8.4 

5.6 

Joint  Primary  Aircraft  Training  System  (JPATS) 

5,999 

3,918 

53.1 

-0.8 

-3.8 

Joint  Standoff  Weapon  -  Baseline  Variant  and  Unitary  Warhead 
Variant  (JSOW) 

5,739 

8,359 

-31.3 

-0.6 

10.7 

JSOW  -  Baseline  Variant 

2,353 

3,004 

-21.7 

-0.2 

0.1 

JSOW  -  Unitary  Variant 

3,386 

5,354 

-36.8 

-0.9 

19.6 

Joint  Tactical  Networks  (JTN) 

2,261 

1,032 

119.1 

3.9 

2.2 

Page  161 


GAO-14-340SP  Assessments  of  Selected  Weapon  Programs 


Appendix  II 

Current  and  First  Full  Estimates  for  DOD’s 
2013  Portfolio  of  Major  Defense  Acquisition 
Programs 


(Continued  From  Previous  Page) 


Fiscal  year  2014  dollars  in  millions 


Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit 
Radios  (JTRS  HMS) 

8,936 

10,557 

-15.4 

2.1 

171.5 

KC-130J  Transport  Aircraft  (KC-130J) 

9,849 

9,975 

-1.3 

-2.2 

-1.3 

KC-46  Tanker  Modernization  Program  (KC-46) 

44,519 

46,090 

-3.4 

-2.7 

-3.4 

LHA  6  America  Class  Amphibious  Assault  Ship 

10,110 

3,345 

202.3 

-2.7 

182.5 

Light  Utility  Helicopter  (LUH),  UH-72A  Lakota 

1,891 

1,904 

-0.7 

-9.3 

-10.6 

Littoral  Combat  Ship  (LCS) — Seaframes 

30,027 

2,360 

NA 

-9.4 

NA 

LPD  1 7  San  Antonio  Class  Amphibious  Transport  Dock  (LPD  1 7) 

19,424 

12,318 

57.7 

-0.3 

23.6 

MH-60R  Multi-Mission  Helicopter  (MH-60R) 

14,358 

5,822 

146.6 

-5.6 

9.5 

MH-60S  Fleet  Combat  Support  Helicopter  (MH-60S) 

8,727 

3,690 

136.5 

-1.2 

0.7 

Mobile  User  Objective  System  (MUOS) 

7,448 

7,069 

5.4 

0.7 

6.7 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (MQ-1C  Gray  Eagle) 

4,951 

1,068 

363.6 

2.1 

94.3 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

13,544 

13,512 

0.2 

0.1 

0.2 

MQ-8  Fire  Scout 

2,811 

2,751 

2.2 

3.1 

29.0 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

12,706 

2,774 

358.1 

-0.4 

358.1 

Multifunctional  Information  Distribution  System  (MIDS) 

3,809 

1,371 

177.7 

8.2 

32.4 

National  Airspace  System  (NAS) 

1,671 

913 

82.9 

-1.7 

-2.1 

Navstar  Global  Positioning  System  (Navstar  GPS) 

9,695 

7,578 

27.9 

0.0 

-4.1 

Navstar  GPS  Space  &  Control 

8,081 

6,538 

23.6 

-0.1 

4.0 

Navstar  GPS  User  Equipment 

1,614 

1,040 

55.2 

0.8 

-30.9 

Navy  Multiband  Terminal  Satellite  (NMT) 

1,937 

2,441 

-20.7 

-0.5 

-9.0 

Next  Generation  Operational  Control  System  (GPS  OCX) 

3,470 

3,482 

-0.3 

-0.3 

-0.3 

P-8A  Poseidon  (P-8A) 

34,336 

32,640 

5.2 

-0.1 

6.4 

Paladin  Integrated  Management  (PIM) 

6,930 

7,033 

-1.5 

-1.3 

-1.5 

Patriot  Advanced  Capability-3  (PAC-3) 

13,157 

5,482 

140.0 

1.6 

23.6 

Patriot/Medium  Extended  Air  Defense  System  Combined  Aggregate 
Program  (Patriot/M EADS  CAP) 

11,745 

28,029 

-58.1 

-1.0 

-56.3 

Patriot/MEADS  CAP  Fire  Unit 

3,383 

20,365 

-83.4 

-0.5 

-82.6 

Patriot/MEADS  CAP  Missile 

8,363 

7,664 

9.1 

-1.2 

12.6 

Remote  Minehunting  System  (RMS) 

1,463 

1,516 

-3.5 

-0.5 

-8.8 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System 
(RQ-4A/B  Global  Hawk) 

9,874 

5,671 

74.1 

-3.6 

-6.5 

Ship  to  Shore  Connector  (SSC) 

4,053 

4,158 

-2.5 

-2.5 

-2.5 
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Fiscal  year  2014  dollars  in  millions 

Program  name 

Current 

total 

acquisition 

cost 

First  full 
estimate 
total 
acquisition 
cost 

Change  in 
total 
acquisition 
cost  from 
first  full 
estimate 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past 
year 
(percent) 

Change  in 
total 
acquisition 
cost  within 
the  past  5 
years 
(percent) 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

3,839 

4,945 

-22.4 

-2.8 

-22.4 

Space  Based  Infrared  System  High  Component  (SBIRS  High) 

19,082 

4,835 

294.7 

-1.0 

43.3 

SSN  774  Virginia  Class  Submarine  (SSN  774) 

84,350 

63,582 

32.7 

-2.6 

-5.0 

Standard  Missile-6  (SM-6) 

8,943 

5,995 

49.2 

41.9 

54.7 

Tactical  Tomahawk  RGM-109E/UGM  109E  Missile 
(Tactical  Tomahawk) 

7,451 

2,226 

234.8 

-1.5 

52.1 

Trident  II  (D-5)  Sea-Launched  Ballistic  Missile  UGM  133A 
(Trident  II  Missile) 

56,419 

54,405 

3.7 

1.1 

4.4 

UH-60M  Black  Hawk  Helicopter  (UH-60M  Black  Hawk) 

24,368 

13,641 

78.6 

-11.1 

7.4 

V-22  Osprey  Joint  Services  Advanced  Vertical  Lift  Aircraft  (V-22) 

60,668 

42,173 

43.9 

1.3 

0.3 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

5,046 

3,900 

29.4 

-20.9 

29.4 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

15,414 

17,214 

-10.5 

17.7 

-10.5 

Wideband  Global  SATCOM  (WGS) 

4,154 

1,255 

230.8 

-1.3 

84.0 

Source:  GAO  analysis  of  DOD  data. 


Notes:  Data  were  obtained  from  DOD’s  Selected  Acquisition  Reports,  acquisition  program  baselines, 
and,  in  some  cases,  program  offices.  Changes  in  total  acquisition  cost  for  the  Littoral  Combat  Ship — 
Seaframes  over  the  past  5  years  and  from  its  first  full  estimate  are  shown  as  “NA”  because  DOD 
reported  an  incomplete  baseline  and  cost  data  for  the  program  through  2010. 
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Changes  in  DOD’s  2013  Portfolio  of  Major 
Defense  Acquisition  Programs  over  5  Years 
and  Since  First  Full  Estimates 


Table  9  shows  the  change  in  research  and  development  cost,  procurement 
cost,  total  acquisition  cost,  and  average  delay  in  delivering  initial 
operational  capability  for  those  programs  in  Department  of  Defense’s 
(DOD)  201 3  portfolio  over  the  last  5  years  and  since  their  first  full  cost  and 
schedule  estimates. 


Table  9:  Cost  and  Schedule  Changes  for  Programs  in  DOD’s  2013  Portfolio 

Fiscal  year  2014  dollars 

5  year  comparison 
(2008  to  2013) 

Since  first  full  estimate 
(Baseline  to  2013) 

Change  in  total  research  and 
development  cost 

$32  billion 
12% 

$98  billion 
51% 

Change  in  total  procurement  cost 

$170  billion 
16% 

$348  billion 
40% 

Change  in  total  other  acquisition  costs3 

$4  billion 
50% 

$2  billion 
18% 

Change  in  total  acquisition  cost 

$207  billion 
16% 

$448  billion 
42% 

Average  delay  in  delivering  initial 
capabilities 

12  months 
14% 

28  months 
36% 

Source:  GAO  analysis  of  DOD  data. 


Notes:  Data  were  obtained  from  DOD’s  SARs  and  acquisition  program  baselines.  In  a  few  cases  data 
were  obtained  directly  from  program  offices.  Some  numbers  may  not  sum  due  to  rounding. 

aln  addition  to  research  and  development  and  procurement  costs,  total  acquisition  cost  includes 
acquisition-related  operation  and  maintenance  and  system-specific  military  construction  costs. 
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Knowledge-Based  Acquisition  Practices 


GAO’s  prior  work  on  best  product-development  practices  found  that 
successful  programs  take  steps  to  gather  knowledge  that  confirms  that 
their  technologies  are  mature,  their  designs  stable,  and  their  production 
processes  are  in  control.  Successful  product  developers  ensure  a  high 
level  of  knowledge  is  achieved  at  key  junctures  in  development.  We 
characterize  these  junctures  as  knowledge  points.  The  Related  GAO 
Products  section  of  this  report  includes  references  to  the  body  of  work  that 
helped  us  identify  these  practices  and  apply  them  as  criteria  in  weapon 
system  reviews.  The  following  table  summarizes  these  knowledge  points 
and  associated  key  practices. 


Table  10:  Best  Practices  for  Knowledge-based  Acquisitions 


Knowledge  Point  1:  Technologies,  time,  funding,  and  other  resources  match 
customer  needs.  Decision  to  invest  in  product  development 

Demonstrate  technologies  to  a  high  readiness  level — Technology  Readiness  Level  7 — to 
ensure  technologies  will  work  in  an  operational  environment3 

Ensure  that  requirements  for  product  increment  are  informed  by  preliminary  design 
review  using  systems  engineering  process  (such  as  prototyping  of  preliminary  design) 

Establish  cost  and  schedule  estimates  for  product  on  the  basis  of  knowledge  from 
preliminary  design  using  systems  engineering  tools  (such  as  prototyping  of  preliminary 
design) 

Constrain  development  phase  (5  to  6  years  or  less)  for  incremental  development 
Ensure  development  phase  fully  funded  (programmed  in  anticipation  of  milestone) 

Align  program  manager  tenure  to  complete  development  phase 

Contract  strategy  that  separates  system  integration  and  system  demonstration  activities 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  for  development  start 

Knowledge  Point  2:  Design  is  stable  and  performs  as  expected.  Decision  to  start 

building  and  testing  production-representative  prototypes 

Complete  system  critical  design  review 

Complete  90  percent  of  engineering  design  drawing  packages 

Complete  subsystem  and  system  design  reviews 

Demonstrate  with  system-level  integrated  prototype  that  design  meets  requirements 
Complete  the  failure  modes  and  effects  analysis 
Identify  key  system  characteristics 
Identify  critical  manufacturing  processes 

Establish  reliability  targets  and  growth  plan  on  the  basis  of  demonstrated  reliability  rates 
of  components  and  subsystems 
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Knowledge-Based  Acquisition  Practices 


(Continued  From  Previous  Page) 

Conduct  independent  cost  estimate 
Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  to  enter  system  demonstration 

Knowledge  Point  3:  Production  meets  cost,  schedule,  and  quality  targets.  Decision 
to  produce  first  units  for  customer 

Demonstrate  manufacturing  processes 

Build  and  test  production-representative  prototypes  to  demonstrate  product  in  intended 
environment 

Test  production-representative  prototypes  to  achieve  reliability  goal 
Collect  statistical  process  control  data 

Demonstrate  that  critical  processes  are  capable  and  in  statistical  control 

Conduct  independent  cost  estimate 

Conduct  independent  program  assessment 

Conduct  major  milestone  decision  review  to  begin  production 

Source:  GAO. 

Note:  DOD  considers  Technology  Readiness  Level  6,  demonstrations  in  a  relevant  environment,  to  be 
appropriate  for  programs  entering  system  development;  therefore  we  have  analyzed  programs  against 
this  measure  as  well. 
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Technology  readiness  level  Description 

1 .  Basic  principles  Lowest  level  of  technology  readiness, 

observed  and  reported  Scientific  research  begins  to  be 

translated  into  applied  research  and 
development.  Examples  might  include 
paper  studies  of  a  technology’s  basic 
properties 

2.  Technology  concept  Invention  begins.  Once  basic 

and/or  application  principles  are  observed,  practical 
formulated  applications  can  be  invented.  The 

application  is  speculative  and  there  is 
no  proof  or  detailed  analysis  to 
support  the  assumption.  Examples  are 
still  limited  to  paper  studies. 

3.  Analytical  and  Active  research  and  development  is 

experimental  critical  initiated.  This  includes  analytical 
function  and/or  studies  and  laboratory  studies  to 

characteristic  proof  of  physically  validate  analytical 
concept  predictions  of  separate  elements  of 

the  technology.  Examples  include 
components  that  are  not  yet  integrated 
or  representative. 


4.  Component  and/or  Basic  technological  components  are 
breadboard  validation  integrated  to  establish  that  the  pieces 
in  laboratory  will  work  together.  This  is  relatively 

environment  “low  fidelity”  compared  to  the  eventual 

system.  Examples  include  integration 
of  “ad  hoc”  hardware  in  a  laboratory. 


5.  Component  and/or  Fidelity  of  breadboard  technology 
breadboard  validation  increases  significantly.  The  basic 
in  relevant  technological  components  are 

environment  integrated  with  reasonably  realistic 

supporting  elements  so  that  the 
technology  can  be  tested  in  a 
simulated  environment.  Examples 
include  “high  fidelity”  laboratory 
integration  of  components. 
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Demonstration 

Hardware/software  environment 

None  (paper  studies  and  analysis)  None 


None  (paper  studies  and  analysis)  None 


Analytical  studies  and  Lab 

demonstration  of  nonscale 
individual  components  (pieces  of 
subsystem) 


Low-fidelity  breadboard.  Lab 

Integration  of  nonscale 
components  to  show  pieces  will 
work  together.  Not  fully  functional 
or  form  or  fit  but  representative  of 
technically  feasible  approach 
suitable  for  flight  articles. 

High-fidelity  breadboard.  Lab  demonstrating 

Functionally  equivalent  but  not  functionality  but  not  form 

necessarily  form  and/or  fit  (size  and  fit.  May  include  flight 
weight,  materials,  etc).  Should  be  demonstrating  breadboard 
approaching  appropriate  scale.  in  surrogate  aircraft. 

May  include  integration  of  several  Technology  ready  for 

components  with  reasonably  detailed  design  studies, 

realistic  support 
elements/subsystems  to 
demonstrate  functionality. 
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(Continued  From  Previous  Page) 

Technology  readiness  level  Description 


Demonstration 

Hardware/software  environment 


6. 


System/subsystem  Representative  model  or  prototype 
model  or  prototype  system,  which  is  well  beyond  the 
demonstration  in  a  breadboard  tested  for  TRL  5,  is  tested 
relevant  environment  in  a  relevant  environment.  Represents 
a  major  step  up  in  a  technology’s 
demonstrated  readiness.  Examples 
include  testing  a  prototype  in  a  high 
fidelity  laboratory  environment  or  in 
simulated  realistic  environment. 


Prototype.  Should  be  very  close  to  High-fidelity  lab 
form,  fit  and  function.  Probably  demonstration  or 
includes  the  integration  of  many  limited/restricted  flight 
new  components  and  realistic  demonstration  for  a 
supporting  elements/subsystems  if  relevant  environment, 
needed  to  demonstrate  full  Integration  of  technology  is 

functionality  of  the  subsystem.  well  defined. 


7.  System  prototype  Prototype  near  or  at  planned  Prototype.  Should  be  form,  fit  and  Flight  demonstration  in 

demonstration  in  a  operational  system.  Represents  a  function  integrated  with  other  key  representative  realistic 

realistic  environment  major  step  up  from  TRL  6,  requiring  supporting  elements/subsystems  environment  such  as  flying 

the  demonstration  of  an  actual  system  to  demonstrate  full  functionality  of  test  bed  or  demonstrator 

prototype  in  a  realistic  environment,  subsystem.  aircraft.  Technology  is  well 

such  as  in  an  aircraft,  vehicle  or  substantiated  with  test 

space.  Examples  include  testing  the  data, 

prototype  in  a  test  bed  aircraft. 


8.  Actual  system  Technology  has  been  proven  to  work  Flight-qualified  hardware  Developmental  Test  and 

completed  and  “flight  in  its  final  form  and  under  expected  Evaluation  (DT&E)  in  the 

qualified”  through  test  conditions.  In  almost  all  cases,  this  actual  system  application, 

and  demonstration  TRL  represents  the  end  of  true  system 
development.  Examples  include 
developmental  test  and  evaluation  of 
the  system  in  its  intended  weapon 
system  to  determine  if  it  meets  design 


specifications. 


9.  Actual  system  “flight  Actual  application  of  the  technology  in  Actual  system  in  final  form 
proven”  through  its  final  form  and  under  mission 

successful  mission  conditions,  such  as  those  encountered 
operations  in  operational  test  and  evaluation.  In 

almost  all  cases,  this  is  the  end  of  the 
last  “bug  fixing”  aspects  of  true  system 
development.  Examples  include  using 
the  system  under  operational  mission 
conditions. 


Operational  Test  and 
Evaluation  (OT&E)  in 
operational  mission 
conditions. 


Source:  GAO  and  its  analysis  of  National  Aeronautics  and  Space  Administration  data. 
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Major  Defense  Acquisition  Programs’ 
Individual  Subcontracting  Reports  in  the 
Electronic  Subcontracting  Reporting  System 


Table  11  contains  the  numbers  of  individual  subcontracting  reports  from 
the  prime  contractor  for  the  programs  we  assessed  that  were  accepted  on 
the  Electronic  Subcontracting  Reporting  System  (eSRS).  We  reviewed  this 
information  for  the  36  major  defense  acquisition  programs  included  in  our 
individual  program  assessments  using  the  prime  contracts  reported  in 
available  Selected  Acquisition  Reports.  The  government  uses  individual 
subcontracting  reports  on  eSRS  as  one  method  of  monitoring  small 
business  participation,  as  the  report  includes  goals  for  small  business 
subcontracting.  There  are  multiple  reasons  why  a  program  may  not  have 
an  accepted  subcontracting  report  in  eSRS.  For  example,  some  programs 
may  have  pending  or  rejected  reports  within  the  system  as  all  reports  are 
reviewed  prior  to  acceptance.  Not  all  prime  contracts  for  major  defense 
acquisition  programs  are  required  to  submit  individual  subcontracting 
reports.  Instead,  some  contractors  report  small  business  participation  at  a 
corporate  level  as  opposed  to  the  program  level  and  this  data  is  not 
captured  in  the  individual  subcontracting  reports.1  Specifically,  the  F-35 
program  provided  questionnaire  data  indicating  that  for  the  five  contracts 
listed  in  the  December  2012  SAR  all  report  at  the  comprehensive, 
contractor  level.  Similarly  for  the  three  contracts  listed  by  JLTV  program 
officials,  questionnaire  data  shows  that  one  of  them  reports  on  small 
business  participation  at  the  corporate  level  with  no  participation 
information  specific  to  the  JLTV  program.  Finally,  although  a  program  may 
be  required  to  submit  a  report  it  may  not  yet  have  done  so  for  the  period 
we  reviewed. 


Twelve  large  defense  contractors  are  participants  under  the  Test  Program  for  Negotiation 
of  Comprehensive  Small  Business  Subcontracting  Plans  created  by  the  National  Defense 
Authorization  Act  for  Fiscal  Years  1990  and  1991,  Pub.  L.  No.  101-189,  §  834  (1989)  and 
have  each  established  a  comprehensive  subcontracting  plan  on  a  corporate,  division  or 
plant-wide  basis  under  which  a  single  summary  subcontract  report  is  submitted  semi¬ 
annually  for  all  covered  DOD  contracts.  The  test  program  has  been  extended  by  Congress 
several  times  with  the  current  three  year  extension  made  by  Pub.  L.  No.  112-81,  §  866 
(2011)  to  end  on  December  31, 2014.  Participation  in  the  test  program  is  on  a  voluntary 
basis  such  that  the  12  participants  may  have  contracts  where  they  are  reporting  on  an 
individual  basis  as  well  as  contracts  where  they  are  reporting  on  a  comprehensive  basis. 
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Major  Defense  Acquisition  Programs’ 

Individual  Subcontracting  Reports  in  the 

Electronic  Subcontracting  Reporting  System 

Table  11:  Major  Defense  Acquisition  Programs’  Individual  Subcontracting  Reports  in  the  Electronic  Subcontracting  Reporting 
System 

Number  of  contracts 
listed  in  the 

Program  name  December  2012  SAR 

Contracts  with  an  accepted 
individual  subcontracting 
report  (as  of  January  2014) 

AIM-9X  Block  II  Air-to-Air  Missile  (AIM-9X  Block  II) 

2 

0 

Airborne  and  Maritime/Fixed  Station  Joint  Tactical  Radio  System  (AMF  JTRS) 

1 

0 

CH-53K  Heavy  Lift  Replacement  Helicopter  (CH-53K) 

1 

0 

Gerald  R.  Ford  Class  Nuclear  Aircraft  Carrier  (CVN  78) 

6 

3 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

4 

3 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

4 

3 

Evolved  Expendable  Launch  Vehicle  (EELV) 

10 

4 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

2 

0 

F-22  Increment  3.2B 

2 

0 

F-35  Lightning  II  (F-35) 

6 

0 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

2 

0 

Global  Positioning  System  III  (GPS  III) 

1 

0 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

1 

0 

Integrated  Air  and  Missile  Defense  (IAMD) 

3 

1 

Joint  Air-to-Surface  Standoff  Missile-Extended  Range  (JASSM-ER) 

3 

0 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System 
(JLENS) 

1 

0 

Joint  Light  Tactical  Vehicle  (JLTV) 

3 

2 

Joint  Precision  Approach  and  Landing  System  Increment  1 A  (JPALS  Inc  1  A) 

1 

0 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios 
(JTRS  HMS) 

1 

0 

KC-46  Tanker  Modernization  Program  (KC-46) 

1 

1 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

2 

2 

Littoral  Combat  Ship  (LCS) — Seaframes 

3 

2 

MQ-1C  Gray  Eagle  Unmanned  Aircraft  System  (MQ-1C  Gray  Eagle) 

6 

5 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

1 

0 

MQ-8  Fire  Scout 

1 

1 

MQ-9  Reaper  Unmanned  Aircraft  System  (MQ-9  Reaper) 

2 

0 

Mobile  User  Objective  System  (MUOS) 

1 

0 

Next  Generation  Operational  Control  System  (GPS  OCX) 

1 

0 

P-8A  Poseidon  (P-8A) 

2 

1 

Paladin  Integrated  Management  (PIM) 

1 

1 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System  (RQ-4A/B  Global  Hawk) 

5 

2 

Ship  to  Shore  Connector  (SSC) 

1 

1 
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(Continued  From  Previous  Page) 

Program  name 

Number  of  contracts 
listed  in  the 
December  2012  SAR 

Contracts  with  an  accepted 
individual  subcontracting 
report  (as  of  January  2014) 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

1 

0 

Space  Based  Infrared  System  High  Component  (SBIRS  High) 

3 

0 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

1 

0 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

1 

0 

Total 

87 

32 

Source:  GAO  analysis  of  data  from  DOD  and  eSRS. 
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Comments  from  the  Department  of  Defense 


OFFICE  OF  THE  UNDER  SECRETARY  OF  DEFENSE 

3000  DEFENSE  PENTAGON 
WASHINGTON.  DC  20301-3000 


f  1  WR 

Mr.  Michael  J.  Sullivan 

Director.  Acquisition  and  Sourcing  Management 
U.S.  Government  Accountability  Office 
441  GStNW 
Washington,  DC  20548 

Dear  Mr.  Sullivan, 

This  is  the  Department  of  Defense  response  to  the  GAO  Draft  Report.  GAO-14-340SP. 
“Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs"  dated  March  2014 
(GAO  Code  121144). 

The  Department  is  pleased  that  this  year's  Draft  Report,  the  12th  annual  assessment  on  the 
performance  of  DoD's  major  acquisition  programs,  is  the  most  encouraging  to  date. 

GAO  results  indicate  that  the  Department's  commitment  to  acquisition  excellence  in  the 
Better  Buying  Power  initiatives  continue  to  have  a  positive  impact  on  cost,  schedule,  and 
performance  measures  of  our  programs. 

The  Under  Secretary  of  Defense  for  Acquisition.  Technology,  and  Logistics  will  again 
publish  a  comprehensive  report  on  the  acquisition  system  in  2014.  The  2nd  annual  report 
will  be  a  data  driven  analysis  that  provides  transparent  and  objective  performance 
measures  of  our  major  acquisition  programs  and  contracts.  The  fiscal  environment 
remains  challenging  and  continues  to  put  downward  pressure  on  investment  funding. 
Continued  improvement  of  the  acquisition  system  will  require  a  more  clear  understanding 
of  how  decisions  throughout  the  process  influence  performance. 

The  Department  appreciates  the  opportunity  to  comment  on  the  Draft  Report.  My  point 
of  contact  for  this  effort  is  Mr.  Joe  Beauregard.  703-697-8046. 


Sincerely. 


Nancy  L.  Spruill 
Director 

Acquisition  Resources  &  Analysis 


ACQUISITION. 
TECHNOLOGY 
AND  LOGISTICS 
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Burns  C.  Eckert,  Jenny  Shinn 

DDG  1000  Zumwalt  Class  Destroyer  (DDG  1000) 

Patrick  M.  Breiding,  Angie  Nichols-Friedman 

DDG  51  Arleigh  Burke  Class  Guided  Missile  Destroyer  (DDG  51) 

Greg  Campbell,  loan  T.  Ifrim 

Evolved  Expendable  Launch  Vehicle  (EELV) 

Desiree  E.  Cunningham,  Maria  A.  Durant 

Excalibur  Precision  155mm  Projectiles  (Excalibur) 

Ryan  Stott,  Carrie  W.  Rogers 

F-22  Increment  3.2B 

Robert  P.  Bullock,  Christopher  D.  Zbrozek, 

Sean  C.  Seales 

F-35  Lightning  II  (F-35) 

Abby  C.  Volk,  Marvin  E.  Bonner 

Family  of  Advanced  Beyond  Line-of-Sight  Terminals  (FAB-T) 

Scott  M.  Purdy,  Alexandra  Dew  Silva 

Global  Positioning  System  III  (GPS  III) 

Laura  T.  Holliday,  Erin  R.  Cohen 

Ground/Air  Task  Oriented  Radar  (G/ATOR) 

Bonita  J.  P.  Oden,  Jonathan  Mulcare 

Integrated  Air  and  Missile  Defense  (IAMD) 

John  M.  Ortiz,  Carol  T.  Mebane 

Joint  Air-to-Surface  Standoff  Missile  Extended  Range  (JASSM-ER) 

Brian  A.  Tittle,  John  W.  Crawford 
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Primary  staff 

Joint  Land  Attack  Cruise  Missile  Defense  Elevated  Netted  Sensor  System  (JLENS) 

Maria  A.  Durant,  John  M.  Ortiz 

Joint  Light  Tactical  Vehicle  (JLTV) 

Dayna  L.  Foster,  Andrea  M.  Bivens 

Joint  Precision  Approach  and  Landing  System  Increment  1A  (JPALS  Inc  1A) 

Stephen  V.  Marchesani,  Jared  A.  Sippel 

Joint  Tactical  Radio  System  Handheld,  Manpack,  and  Small  Form  Fit  Radios 
(JTRS  HMS) 

Nathan  A.  Tranquilli,  Aryn  C.  Ehlow, 

Paul  G.  Williams 

KC-46  Tanker  Modernization  Program  (KC-46) 

Jeffrey  L.  Hartnett,  Katheryn  S.  Hubbell 

LHA  6  America  Class  Amphibious  Assault  Ship  (LHA  6) 

Christopher  E.  Kunitz,  Jenny  Chow 

Littoral  Combat  Ship  (LCS) — Mission  Packages 

LaurierR.  Fish,  Greg  Campbell 

Littoral  Combat  Ship  (LCS) — Seaframes 

Amber  N.  Keyser,  Christopher  R.  Durbin 

Mobile  User  Objective  System  (MUOS) 

Richard  Y.  Horiuchi,  Desiree  E.  Cunningham 

MQ-1C  Unmanned  Aircraft  System  (MQ-1C  Gray  Eagle) 

Deanna  Laufer,  James  S.  Kim 

MQ-4C  Triton  Unmanned  Aircraft  System  (MQ-4C  Triton) 

Tom  Twambly,  Candice  N.  Wright, 

Amanda  Parker 

MQ-8  Fire  Scout 

Leigh  Ann  Haydon,  Deanna  Laufer 

MQ-9  Unmanned  Aircraft  System  Reaper  (MQ-9  Reaper) 

Megan  L.  Porter,  Laura  Jezewski 

Next  Generation  Operational  Control  System  (GPS  OCX) 

Erin  R.  Cohen,  Jeffrey  M.  Sanders 

P-8A  Poseidon  (P-8A) 

Jocelyn  C.  Yin,  Heather  B.  Miller 

Paladin  Integrated  Management  (PIM) 

William  C.  Allbritton,  Marcus  C.  Ferguson 

RQ-4A/B  Global  Hawk  Unmanned  Aircraft  System  (RQ-4A/B  Global  Hawk) 

Laura  M.  Jezewski,  Megan  L.  Porter 

Ship  to  Shore  Connector  (SSC) 

Matthew  Shaffer,  Beth  Reed  Fritts 

Small  Diameter  Bomb  Increment  II  (SDB  II) 

John  W.  Crawford,  Brian  A.  Tittle 

Space  Based  Infrared  System  High  Component  (SBIRS  High) 

Claire  A.  Buck,  Maricela  Cherveny 

Warfighter  Information  Network-Tactical  Increment  2  (WIN-T  Inc  2) 

James  P.  Tallon,  Andrea  Yohe 

Warfighter  Information  Network-Tactical  Increment  3  (WIN-T  Inc  3) 

James  P.  Tallon,  Andrea  Yohe 

Future  Programs 

Amphibious  Combat  Vehicle  (ACV) 

MacKenzie  H.  Cooper,  Bonita  J.  P.  Oden 

Armored  Multi-Purpose  Vehicle  (AMPV) 

Andrea  M.  Bivens,  Dayna  L.  Foster 

B-2  Defensive  Management  System  (DMS)  Modernization 

Don  M.  Springman,  Matthew  B.  Lea 

Combat  Rescue  Helicopter  (CRH) 

J.  Andrew  Walker,  Robert  K.  Miller 

Common  Infrared  Countermeasures  (CIRCM) 

Danny  G.  Owens,  Wendy  P.  Smythe 

Enhanced  Polar  System  (EPS) 

Bradley  L.  Terry 

F-15  Eagle  Passive/Active  Warning  and  Survivability  System 
(F-15  EPAWSS) 

Wendell  K.  Hudson,  LeAnna  M.  Parkey 

Ground  Combat  Vehicle  (GCV) 

Marcus  C.  Ferguson,  William  C.  Allbritton 

Indirect  Fire  Protection  Capability,  Increment  2  (IFPC  Inc  2) 

Helena  Brink,  Carol  T.  Mebane 

Joint  Air-to-Ground  Missile  (JAGM) 

Wendy  P.  Smythe,  Ryan  Stott 
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Military  GPS  User  Equipment  (MGUE),  Increment  1 

Raj  C.  Chitikila,  Andrew  H.  Redd 

Next  Generation  Jammer  (NGJ) 

Teakoe  S.  Coleman,  Laura  T.  Holliday 

Ohio  Class  Replacement  (OR) 

C.  James  Madar,  Amber  N.  Keyser 

Presidential  Aircraft  Recapitalization  Program  (PAR) 

LeAnna  M.  Parkey,  Brian  Smith 

Presidential  Helicopter  (VXX) 

Bonita  J.  P.  Oden,  Jerry  W.  Clark 

Space  Fence 

Laura  D.  Hook,  Jeffrey  M.  Sanders 

Three-Dimensional  Expeditionary  Long-Range  Radar  (3DELRR) 

Claire  Li,  Janet  L.  McKelvey 

Unmanned  Carrier-Launched  Airborne  Surveillance  and  Strike  (UCLASS)  System 

Julie  C.  Hadley,  Sean  D.  Merrill 
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Defense  Acquisitions:  Where  Should  Reform  Aim  Next?  GAO-1 4-1 45T. 
Washington,  D.C.:  October  29,  2013. 

Ford-Class  Carriers:  Lead  Ship  Testing  and  Reliability  Shortfalls  Will  Limit 
Initial  Fleet  Capabilities.  GAO-13-396.  Washington,  D.C.: 

September  5,  2013. 

Navy  Shipbuilding:  Significant  Investments  in  the  Littoral  Combat  Ship 
Continue  Amid  Substantial  Unknowns  about  Capabilities,  Use,  and  Cost. 
GAO-13-530.  Washington,  D.C.:  July  22,  2013. 

Missile  Defense:  Opportunity  to  Refocus  on  Strengthening  Acquisition 
Management.  GAO-13-432.  Washington,  D.C.:  April  26,  2013. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs. 
GAO-1 3-294SP.  Washington,  D.C.:  March  28,  2013. 

High-Risk  Series:  An  Update.  GAO-13-283.  Washington,  D.C.: 

February  2013. 

Weapons  Acquisition  Reform:  Reform  Act  Is  Helping  DOD  Acquisition 
Programs  Reduce  Risk,  but  Implementation  Challenges  Remain. 
GAO-13-103.  Washington,  D.C.:  December  14,  2012. 

Joint  Strike  Fighter:  DOD  Actions  Needed  to  Further  Enhance 
Restructuring  and  Address  Affordability  Risks.  GAO-12-437.  Washington, 
D.C.:  June  14,  2012. 

Missile  Defense:  Opportunity  Exists  to  Strengthen  Acquisitions  by 
Reducing  Concurrency.  GAO-12-486.  Washington,  D.C.:  April  20,  2012. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs. 

GAO-1 2-400SP.  Washington,  D.C.:  March  29,  2012. 

Arleigh  Burke  Destroyers:  Additional  Analysis  and  Oversight  Required  to 
Support  the  Navy's  Future  Surface  Combatant  Plans.  GAO-12-113, 
Washington,  D.C.:  January  24,  2012. 

Weapons  Acquisition  Reform:  Actions  Needed  to  Address  Systems 
Engineering  and  Developmental  Testing  Challenges.  GAO-1 1-806. 
Washington,  D.C.:  September  19,  2011. 
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Defense  Acquisitions:  DOD  Can  Improve  Its  Management  of  Configuration 
Steering  Boards.  GAO-1 1-640.  Washington,  D.C.:  July  7,  2011. 

Space  and  Missile  Defense  Acquisitions:  Periodic  Assessment  Needed  to 
Correct  Parts  Quality  Problems  in  Major  Programs.  GAO-1 1  -404. 
Washington,  D.C.:  June  24,  2011. 

Defense  Acquisitions:  Assessments  of  Selected  Weapon  Programs.  GAO- 
11-233SP.  Washington,  D.C.:  March  29,  2011. 

Defense  Acquisitions:  Observations  on  Weapon  Program  Performance 
and  Acquisition  Reforms.  GAO-10-706T.  Washington,  D.C:  May  19,  2010. 

Defense  Acquisitions:  Strong  Leadership  Is  Key  to  Planning  and  Executing 
Stable  Weapon  Programs.  GAO-10-522.  Washington,  D.C.:  May  6,  2010. 

Best  Practices:  DOD  Can  Achieve  Better  Outcomes  by  Standardizing  the 
Way  Manufacturing  Risks  Are  Managed.  GAO-10-439.  Washington,  D.C: 
April  22,  2010. 

Best  Practices:  High  Levels  of  Knowledge  at  Key  Points  Differentiate 
Commercial  Shipbuilding  from  Navy  Shipbuilding.  GAO-09-322. 
Washington,  D.C.:  May  13,  2009. 

Defense  Acquisitions:  Charting  a  Course  for  Lasting  Reform.  GAO-09- 
663T.  Washington,  D.C.:  April  30,  2009. 

Defense  Acquisitions:  Measuring  the  Value  of  DOD’ s  Weapon  Programs 
Requires  Starting  with  Realistic  Baselines.  GAO-09-543T.  Washington, 
D.C.:  April  1, 2009. 

Defense  Acquisitions:  DOD  Must  Balance  Its  Needs  with  Available 
Resources  and  Follow  an  Incremental  Approach  to  Acquiring  Weapon 
Systems.  GAO-09-431T.  Washington,  D.C.:  March  3,  2009. 

GAO  Cost  Estimating  and  Assessment  Guide:  Best  Practices  for 
Developing  and  Managing  Capital  Program  Costs.  GAO-09-3SP. 
Washington,  D.C.:  March  2,  2009. 
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Defense  Acquisitions:  Perspectives  on  Potential  Changes  to  Department 
of  Defense  Acquisition  Management  Framework.  GAO-09-295R. 
Washington,  D.C.:  February  27,  2009. 

Defense  Acquisitions:  Cost  to  Deliver  Zumwalt-Class  Destroyers  Likely  to 
Exceed  Budget.  GAO-08-804.  Washington,  D.C.:  July  31,  2008. 

Defense  Acquisitions:  A  Knowledge-Based  Funding  Approach  Could 
Improve  Major  Weapon  System  Program  Outcomes.  GAO-08-619. 
Washington,  D.C.:  July  2,  2008. 

Best  Practices:  Increased  Focus  on  Requirements  and  Oversight  Needed 
to  Improve  DOD’s  Acquisition  Environment  and  Weapon  System  Quality. 
GAO-08-294.  Washington,  D.C.:  February  1, 2008. 

Best  Practices:  An  Integrated  Portfolio  Management  Approach  to  Weapon 
System  Investments  Could  Improve  DOD’s  Acquisition  Outcomes. 
GAO-07-388.  Washington,  D.C.:  March  30,  2007. 

Defense  Acquisitions:  Major  Weapon  Systems  Continue  to  Experience 
Cost  and  Schedule  Problems  under  DOD’s  Revised  Policy.  GAO-06-368. 
Washington,  D.C.:  April  14,  2006. 

Defense  Acquisitions:  Improved  Management  Practices  Could  Help 
Minimize  Cost  Growth  in  Navy  Shipbuilding  Programs.  GAO-05-183. 
Washington,  D.C.:  February  28,  2005. 

Best  Practices:  Using  a  Knowledge-Based  Approach  to  Improve  Weapon 
Acquisition.  GAO-04-386SP.  Washington,  D.C.:  January  2004. 

Best  Practices:  Capturing  Design  and  Manufacturing  Knowledge  Early 
Improves  Acquisition  Outcomes.  GAO-02-701.  Washington,  D.C.: 

July  15,  2002. 

Best  Practices:  Better  Matching  of  Needs  and  Resources  Will  Lead  to 
Better  Weapon  System  Outcomes.  GAO-01 -288.  Washington,  D.C.: 

March  8,  2001. 

Best  Practices:  Better  Management  of  Technology  Development  Can 
Improve  Weapon  System  Outcomes.  GAO/NSIAD-99-162.  Washington, 
D.C.:  July  30,  1999. 
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GAO’s  Mission 

The  Government  Accountability  Office,  the  audit,  evaluation,  and 
investigative  arm  of  Congress,  exists  to  support  Congress  in  meeting  its 
constitutional  responsibilities  and  to  help  improve  the  performance  and 
accountability  of  the  federal  government  for  the  American  people.  GAO 
examines  the  use  of  public  funds;  evaluates  federal  programs  and 
policies;  and  provides  analyses,  recommendations,  and  other  assistance 
to  help  Congress  make  informed  oversight,  policy,  and  funding  decisions. 
GAO's  commitment  to  good  government  is  reflected  in  its  core  values  of 
accountability,  integrity,  and  reliability. 

Obtaining  Copies  of 
GAO  Reports  and 
Testimony 

The  fastest  and  easiest  way  to  obtain  copies  of  GAO  documents  at  no  cost 
is  through  GAO's  Web  site  (www.gao.gov).  Each  weekday  afternoon,  GAO 
posts  on  its  Web  site  newly  released  reports,  testimony,  and 
correspondence.  To  have  GAO  e  mail  you  a  list  of  newly  posted  products, 
go  to  www.gao.gov  and  select  "E-mail  Updates." 

Order  by  Phone 

The  price  of  each  GAO  publication  reflects  GAO's  actual  cost  of 
production  and  distribution  and  depends  on  the  number  of  pages  in  the 
publication  and  whether  the  publication  is  printed  in  color  or  black  and 
white.  Pricing  and  ordering  information  is  posted  on  GAO's  Web  site, 

http://www.gao.gov/ordering.htm. 

Place  orders  by  calling  (202)  512-6000,  toll  free  (866)  801-7077,  or 

TDD  (202)512-2537. 

Orders  may  be  paid  for  using  American  Express,  Discover  Card, 
MasterCard,  Visa,  check,  or  money  order.  Call  for  additional  information. 

Connect  with  GAO 

Connect  with  GAO  on  Facebook,  Flickr,  Twitter,  and  YouTube. 

Subscribe  to  our  RSS  Feeds  or  E-mail  Updates.  Listen  to  our  Podcasts. 
Visit  GAO  on  the  web  at  www.gao.gov. 

To  Report  Fraud, 
Waste,  and  Abuse  in 
Federal  Programs 

Contact: 

Web  site:  www.gao.gov/fraudnet/fraudnet.htm 

E-mail:  fraudnet@gao.gov 

Automated  answering  system:  (800)  424-5454  or  (202)  512-7470 

Congressional 

Relations 

Katherine  Siggerud,  Managing  Director,  siggerudk@gao.gov, 

(202)  512-4400  U.S.  Government  Accountability  Office,  441  G  Street  NW, 
Room  7125  Washington,  DC  20548 

Public  Affairs 

Chuck  Young,  Managing  Director,  youngd@gao.gov,  (202)  512-4800 

U.S.  Government  Accountability  Office,  441  G  Street  NW,  Room  7149 
Washington,  DC  20548 

